13. File Identification, Format Detection, and Text Decoding

Detection evaluates bounded evidence in this order: container or magic signature, trusted parser probe, BOM, encoding validity, filename and extension, MIME hint, shebang, modeline, and content heuristic. Signature and successful parser evidence outrank extension and MIME claims. Every result preserves confidence, evidence, conflicts, and any user override.

flowchart TB
  bytes["Bounded source bytes"] --> signature{"Known signature or container?"}
  signature -->|Yes| probe["Run bounded parser probe"]
  signature -->|No| bom["Check BOM and text validity"]
  probe --> evidence["Record high-confidence evidence"]
  bom --> hints["Evaluate filename, MIME, shebang, modeline, and content hints"]
  hints --> evidence
  evidence --> conflict{"Evidence conflicts or budget exhausted?"}
  conflict -->|Yes| safe["Choose safe fallback and show warning"]
  conflict -->|No| renderer["Select registered renderer"]
  renderer --> override["Allow explicit session override"]

Detection reads at most 256 KiB before renderer selection, spends at most 100 ms desktop or 200 ms Android on synchronous evidence, and runs expensive probes in a worker with a 2-second hard timeout. Archive and PDF probes read only structural ranges when seek is available.

.mmd and .mermaid are Mermaid candidate evidence, not authority. A standalone Mermaid result requires a valid text profile plus a bounded supported diagram declaration or valid Mermaid frontmatter followed by a declaration. Ambiguous text opens through the highest-confidence safe renderer and retains an explicit session override.

Text decoding supports UTF-8, UTF-16LE, UTF-16BE, and a reviewed set of legacy encodings through a dedicated decoding library. UTF-8 without BOM is the default only when bytes validate. Invalid-byte replacement marks the profile as not round-trip safe and disables ordinary save until the user selects an encoding or explicitly accepts a lossy save. BOM presence, newline kind per line, and terminal-newline state are preserved. A user normalization command is the only operation that rewrites mixed newline structure.