How we auto-detect the format of pasted text

Drop a file on OmniViewer and we have a filename, an extension and 64 KB of magic-byte-sniffable content to work with. Paste a blob and we have none of that — no name, no extension, maybe not even a complete document, because you might still be typing it. This page is the exact algorithm that turns bare characters into "open this in /json", with the real code.

The problem: text with no name

The OmniViewer front page is an empty buffer: you can drop a file, paste, or just start typing at the cursor. For a dropped file the format decision is easy — the content sniff reads three 64 KB windows, checks magic bytes and text encodings, and the extension picks the toolkit. A paste bypasses all of that. If you copy a JSON API response out of a terminal and paste it, the only evidence of "this is JSON" is the text itself.

And the typed case is harder still: after one keystroke the buffer is {. After five it's {"na. The detector runs on every keystroke, so it must be cheap, and it must be honest about partial input — {"na is not valid JSON, but it is unmistakably a valid prefix of JSON.

Three design rules

1. Assume txt until proven otherwise. A wrong "plain text" verdict is a shrug — the page just stays a buffer. A wrong "this is JSON" verdict yanks you into the /json toolkit with a broken promise. Every detector therefore errs toward txt, and prose full of commas or the word export must never flip the verdict.
2. Accept prefixes. The typed buffer grows one character at a time. Each detector is written so that truncation anywhere — mid-string, mid-number, an unclosed bracket, a tag still being typed — reads as "valid so far", not "invalid".
3. Stay bounded. A paste can be many megabytes. The scan looks at the first 16 KB and the first 40 lines, and never calls JSON.parse — a multi-MB paste classifies in well under a millisecond either way.

The cascade, most reliable signal first

// text-detect.js — the whole decision, top to bottom
export function detectTextFormat(text) {
  const sample  = String(text ?? '').slice(0, SCAN_CHARS).replace(/^/, '');
  const trimmed = sample.trimStart();
  if (!trimmed) return verdict('txt');

  const c = trimmed[0];
  if ((c === '{' || c === '[') && isJsonPrefix(trimmed)) return verdict('json');
  if (c === '<') {
    const markup = detectMarkup(trimmed);            // 'html' | 'xml' | null
    if (markup) return verdict(markup);
  }

  const lines = sampleLines(sample, text.length > SCAN_CHARS);
  const md = markdownScore(lines, trimmed);
  const js = javascriptScore(lines, trimmed);
  if (md >= 3 && md >= js) return verdict('md');
  if (js >= 3) return verdict('js');

  const csv = detectDelimited(lines);                // 'csv' | 'tsv' | null
  if (csv) return verdict(csv);

  return verdict('txt');
}

Structure beats statistics: a leading { that tokenizes as JSON is a stronger signal than any word count, so it's checked first. Markup is next (an unambiguous first tag), then the two "scored" formats that can contain fragments of each other, and finally CSV — the format most easily faked by ordinary prose, which is why it runs last and is the strictest.

JSON: a prefix tokenizer, not a parser

The obvious implementation — try { JSON.parse(text) } — fails both the prefix rule (a half-typed document throws) and the bounded rule (parsing a 50 MB paste to answer "what is this?" is absurd). Instead we walk the sample as a stream of JSON tokens and reject on the first character that could not appear in a valid document at that point. Running out of input is never an error:

const scanString = () => {
  i++;                                    // opening quote
  while (i < n) {
    const ch = s[i];
    if (ch === '\\') { i += 2; continue; }
    if (ch === '"')  { i++; return true; }
    if (ch === '\n' || ch === '\r') return false; // raw newline: not JSON
    i++;
  }
  return true;  // truncated mid-string: still a valid PREFIX
};

The walker tracks what may come next (value, key, colon, comma-or-close) and a bracket stack. That state machine is what cleanly separates JSON from JavaScript object literals:

detect('{"a": 1')      // → json   (valid prefix: key, colon, number)
detect('{')            // → json   (the very first keystroke!)
detect('[1, 2,')       // → json   (truncated array)
detect('{a: 1}')       // → txt    (after '{' only '"' or '}' may follow —
                       //           an unquoted key falls through to the JS scorer)

HTML vs XML: the first tag decides

Anything starting with < gets three cheap checks: <?xml is XML, <!doctype html is HTML, and otherwise the first tag name is looked up in a set of ~60 real HTML tags. <div>, <meta>, <article> → HTML; <catalog>, <record> → XML. The prefix rule shows up here too — a tag cut off by the end of the input is judged by whether it can still become an HTML tag:

// "<di" → html (can complete into <div>); "<catalo" → xml (no HTML tag starts that way)
for (const t of HTML_TAGS) { if (t.startsWith(tag)) return 'html'; }
return 'xml';

A bare < stays txt: one character in, it's just as likely the start of prose like < 5 minutes.

Markdown vs JavaScript: weighted scores, floor of 3

These two are the awkward pair, because each legitimately contains the other: Markdown docs carry fenced ```js code, and JavaScript carries JSDoc comments whose * item lines look exactly like Markdown bullets. Neither gets a single decisive marker, so both accumulate a score over the sampled lines — headings, fences, tables and links for Markdown; declarations, import/require, arrows and comment lines for JavaScript — and the stronger one wins:

if (md >= 3 && md >= js) return verdict('md');
if (js >= 3)             return verdict('js');

The floor of 3 is the txt-bias rule made concrete: one stray “you can export the data” in prose scores below it, and a lone - milk / - eggs shopping list stays plain text. A single # Heading however scores 3 by itself — headings are the most Markdown-specific thing there is, and it means the dropdown flips to Markdown two keystrokes into typing one.

CSV: the strictest test, run last

Prose is full of commas, so CSV must clear the highest bar: every complete sampled line must agree on the same delimiter count — commas, semicolons or tabs — counted outside double-quoted fields:

function countDelims(line, delim) {
  let count = 0, inQuotes = false;
  for (let i = 0; i < line.length; i++) {
    const ch = line[i];
    if (ch === '"') inQuotes = !inQuotes;
    else if (ch === delim && !inQuotes) count++;   // "doe, jane" doesn't count
  }
  return count;
}

Plus two guards: a single-delimiter shape (a,b) needs at least three agreeing lines, since two 1-comma lines could easily be sentences; and blank lines fail the test on purpose — data doesn't have them, paragraphs do. Tab-consistent lines come out as tsv; both land in the /csv toolkit (table view, in-browser SQL, CSV→JSON).

Worked examples

What happens with the verdict

The verdict is only half the feature — the other half is that nothing weird happens with it:

• Paste a recognized format and the blob is wrapped in a synthetic file (pasted.json, pasted.csv, …) and pushed through the exact same loading path as a dropped file — content sniff, windowed raw view, background formatting, the works. You land on /json (or /csv, /md, /js, /html) as if you'd opened a file.
• Paste plain text and the page stays the typed buffer it always was.
• Type, and from the first keystroke a small format: dropdown tracks the live verdict — Text until the buffer looks like something stronger — listing every format so you can jump to the matching toolkit (or override the verdict) with one click.

And per the house rules: all of this runs in your browser. The pasted text is never uploaded — there is no server to upload it to.

{"city": "Paris", "population": 2102650, "landmarks": ["Eiffel Tower", "Louvre"], "river": "Seine"}