Inside a GGUF file: metadata, tensors & quantization

A .gguf file is a whole large language model in a single file — the format llama.cpp, Ollama and LM Studio load. Its best trait, for a viewer, is that it is header-first: everything describing the model sits at the front, before the gigabytes of weights. This is a tour of that header — the magic and version, the key/value metadata block and its value-type enum, the tensor-info table and the GGML quantization types — and how all of it lets you read a model’s entire architecture from a few megabytes, without ever touching the weights.

Why GGUF exists

Training frameworks store models as a folder of tensors plus a pile of JSON config and tokenizer files. That is fine on a training cluster and awkward everywhere else: to run a model you need the weights and the config and the tokenizer, all in agreement. GGUF (the "GGML Universal File") collapses all of that into one self-describing, memory-mappable file. It superseded the older GGML and GGJT formats in 2023, and it is now the standard way quantized models are shipped for local inference. A single .gguf carries the weights, every hyperparameter needed to build the computation graph, and the full tokenizer — so a runner can mmap the file and go.

The file layout: header at the front

Unlike a .dmg (whose index sits at the end), GGUF puts its description right at offset 0. The layout is four regions, in order:

┌───────────────────────────────────────────────┐  offset 0
│  HEADER    magic "GGUF" · version · counts      │
├───────────────────────────────────────────────┤
│  METADATA  key/value pairs (general.*, arch.*,  │
│            tokenizer.* …)                        │
├───────────────────────────────────────────────┤
│  TENSOR    one row per tensor: name, shape,      │
│  INFO      ggml type, offset                     │
├───────────────────────────────────────────────┤
│  PADDING   aligned up to general.alignment (32)  │
├───────────────────────────────────────────────┤
│                                                 │
│  TENSOR DATA   the raw weights — gigabytes —     │
│                never read to inspect the model   │
│                                                 │
└───────────────────────────────────────────────┘

The first three regions are the "header" in the loose sense: a few kilobytes to a few megabytes, even for a model with a 128,000-token vocabulary. Everything a viewer needs is there. The fourth region — the actual weights — is the entire point of the multi-gigabyte file size, and you never need to read a byte of it to describe the model.

The header: magic, version, counts

The file opens with a four-byte magic, the ASCII string GGUF (0x47 0x47 0x55 0x46), then a uint32 version (2 or 3 in the wild; 1 is extinct), then two counts. In version 2 and 3 the counts and every length are uint64; in version 1 they were uint32 — the one difference that threads through the whole parse. Everything is little-endian.

const GGUF_MAGIC = 0x46554747; // "GGUF" read as a little-endian u32

function readHeader(dv) {
  if (dv.getUint32(0, true) !== GGUF_MAGIC) throw new Error('not a GGUF file');
  const version = dv.getUint32(4, true);
  const readLen = version === 1
    ? (o) => dv.getUint32(o, true)          // v1: 32-bit counts/lengths
    : (o) => Number(dv.getBigUint64(o, true)); // v2+: 64-bit
  const tensorCount = readLen(8);
  const kvCount = readLen(version === 1 ? 12 : 16);
  return { version, tensorCount, kvCount };
}

The metadata block: a typed key/value store

After the header come kvCount key/value pairs. Each is a length-prefixed UTF-8 key, a uint32 value-type tag, then the value. The type tag is a small enum:

A string is a uint64 length then that many UTF-8 bytes. An array is a uint32 element-type tag, a uint64 length, then that many values of that type (arrays can even nest). Reading one value is a small recursive switch:

function readValue(r, type, readLen) {
  switch (type) {
    case 4:  return r.u32();
    case 6:  return r.f32();
    case 7:  return r.u8() !== 0;
    case 8:  return r.string(readLen);          // len-prefixed UTF-8
    case 9: {                                    // array
      const elemType = r.u32();
      const n = readLen(r);
      const out = [];
      for (let i = 0; i < n; i++) out.push(readValue(r, elemType, readLen));
      return out;
    }
    // …the remaining scalar widths…
  }
}

Reading the architecture

Metadata keys are namespaced. general.* keys describe identity and provenance; the architecture’s own hyperparameters live under its name, read from general.architecture. So for a Llama model you read llama.block_count, llama.embedding_length, llama.attention.head_count, and so on:

const arch = md['general.architecture'];        // e.g. "llama", "qwen2", "phi3"
const g = (k) => md[`${arch}.${k}`];

const layers   = g('block_count');
const context  = g('context_length');
const heads    = g('attention.head_count');
const kvHeads  = g('attention.head_count_kv');   // < heads ⇒ grouped-query attention
const experts  = g('expert_count');              // > 0 ⇒ Mixture-of-Experts
const usedExp  = g('expert_used_count');         // e.g. 2 of 8 (Mixtral)

The attention type falls straight out of two numbers: when head_count_kv equals head_count it is classic multi-head attention; when it is 1 it is multi-query attention; anything in between is grouped-query attention, and heads / kvHeads is the group ratio. And when expert_count > 1, the model is a Mixture of Experts — Mixtral reports 8 experts with 2 routed per token, and it does so under the llama architecture name, which is why the expert keys, not the architecture, are what you check.

The tensor table

After the metadata come tensorCount tensor-info records. Each is a length-prefixed name, a uint32 dimension count, that many uint64 dimensions, a uint32 GGML type, and a uint64 offset into the data region. This is the table that lets you compute the parameter count directly — no config file required:

let params = 0;
for (let i = 0; i < tensorCount; i++) {
  const name = r.string(readLen);
  const nDims = r.u32();
  const dims = [];
  for (let d = 0; d < nDims; d++) dims.push(readLen(r));
  const ggmlType = r.u32();
  const offset = r.u64();
  params += dims.reduce((a, b) => a * b, 1);     // elements in this tensor
}

Summing the element counts across every tensor gives the model’s parameter count — the number you see advertised as "7B" or "8x7B" — reconstructed from the shapes alone.

Quantization & bits-per-weight

The GGML type on each tensor names its quantization scheme. Quantized types pack weights into fixed-size blocks — a block of block weights occupies size bytes — so the type doubles as an on-disk size formula. A few of the common ones:

From those two numbers you get a tensor’s exact byte size — (elements / block) × size — and, across the file, the model’s effective bits per weight: the whole file’s size in bits divided by the parameter count. That single number tells you how aggressively a model was quantized, straight from the file:

const bitsPerWeight = (fileSize * 8) / params;   // Q4_K models land around 4.5

Reading a 40 GB model from a few megabytes

Because everything above lives at the front of the file, a viewer never has to read the weights. In the browser a dropped File is just a handle; file.slice(0, n) reads only the first n bytes. The one wrinkle is that you don’t know the header’s exact length up front — a big vocabulary can push the tensor table past your first slice — so you read a generous prefix and grow it only if the parse runs off the end:

let prefix = 4 * 1024 * 1024;                    // 4 MB covers almost everything
let buf = new Uint8Array(await file.slice(0, prefix).arrayBuffer());
for (;;) {
  try { return parseGguf(buf, file.size); }       // parse the header
  catch (e) {
    if (e instanceof NeedMoreBytes && prefix < file.size) {
      prefix = Math.min(file.size, Math.max(e.need, prefix * 2));
      buf = new Uint8Array(await file.slice(0, prefix).arrayBuffer());
      continue;                                    // grow and retry
    }
    throw e;
  }
}

The multi-gigabyte weight body is never touched, so a 40 GB model opens as fast as a 40 MB one. That is exactly how OmniViewer’s GGUF viewer works — the parse runs in a Web Worker off a bounded prefix, and nothing is uploaded.

Where the spec lives

GGUF is documented in the ggml repository (docs/gguf.md), with the value-type enum, the standardized general.* and {arch}.* key names and the tokenizer conventions. The GGML tensor types and their block sizes live in the llama.cpp source (ggml.h / ggml.c). Between the two you can parse any GGUF file byte for byte — which is what the code above, and OmniViewer’s reader, do.