Parsing GGUF Metadata for LLaMA-family Models

Full report

On this page

Executive summary

If the goal is to load a real GGUF LLaMA-family model only far enough to parse metadata exactly, the shortest reliable path is to implement a container parser, not an inference engine. GGUF is structurally simple: a fixed header, a sequential metadata key-value region, a sequential tensor-descriptor region, padding to an alignment boundary, and then tensor bytes. For metadata-only work, you do not need to dequantize tensors, build a computation graph, or even touch most of the tensor payload; you only need enough binary logic to parse the header, walk metadata entries recursively, parse tensor descriptors, and compute the tensor_data base address correctly.

The most useful reference implementations today are the GGUF spec in ggml, the gguf Python package and its GGUFReader in llama.cpp, and the Hugging Face Transformers GGUF loader if you also want a high-level cross-check. The gguf package is the best primary-source reference for exact binary parsing and metadata editing; llama.cpp is the best reference for what an executor expects from a valid file; Transformers is useful for compatibility testing, but it is a poor choice if your objective is a lightweight metadata parser because it loads GGUF into a PyTorch-compatible model by dequantizing to higher precision.

Two implementation details matter more than they first appear. First, tensor offsets are relative to the start of tensor_data, not the start of the file, so your parser must finish walking metadata and tensor infos, then align forward using general.alignment or the default of 32, before it can resolve absolute tensor byte positions. Second, version and endianness are tricky: GGUF v3 added big-endian support, but the spec also says there is no definitive in-file way to tell whether a model is big-endian, so parsers either need external knowledge or a guarded heuristic. The shipped Python reader in llama.cpp uses a heuristic based on the version field and supports versions 2 and current, not v1.

For LLaMA-family validation, the minimum schema check is: general.architecture == "llama", required LLaMA-shaped metadata is present under llama.*, tokenizer arrays are internally consistent, and standardized tensor names such as token_embd.weight, output_norm.weight, output.weight, and blk.N.* are present with plausible dimensions. Older artifacts still matter in practice: public models such as older TheBloke TinyLlama GGUFs are explicitly labeled GGUFv2, while the current spec documents v3. A parser meant for “real model parity” should therefore support at least v2 and v3.

That milestone also lines up with the engineering direction you provided: exact GGUF metadata parsing is the first clean, low-level checkpoint on the path from a llama.cpp-dependent stack to a pure C# replacement with stricter binary parity and room for lower-level control logic.

GGUF container anatomy

GGUF is designed as a single-file, extensible, mmap-compatible container that includes the information an executor needs to load a model without external sidecar metadata. In the current spec, the file begins with a 4-byte magic (GGUF), then a version field, then explicit counts for tensors and metadata pairs, then the metadata section, then tensor descriptors, then padding to the required alignment, and finally raw tensor bytes.

flowchart TD
    A[Magic: 4 bytes 'GGUF'] --> B[Version: u32]
    B --> C[Tensor count]
    C --> D[Metadata KV count]
    D --> E[Metadata KV entries]
    E --> F[Tensor info entries]
    F --> G[Pad to general.alignment]
    G --> H[Tensor data blob]

    E1[KV: key string] --> E2[value_type u32]
    E2 --> E3[value payload]
    F1[Tensor name string] --> F2[n_dimensions u32]
    F2 --> F3[dims array]
    F3 --> F4[ggml_type u32]
    F4 --> F5[offset relative to tensor_data]

The exact binary rules that matter most

Metadata keys are not arbitrary strings. The spec requires them to be ASCII, hierarchical, dotted, and based on lower snake case segments. Metadata values are tagged with a gguf_metadata_value_type, and the current standardized on-disk value types are uint8, int8, uint16, int16, uint32, int32, float32, bool, string, array, uint64, int64, and float64. Strings are length-prefixed UTF-8 byte sequences; arrays are homogeneous and may be nested.

One subtle but important correction to common mental models: GGUF metadata does not define a first-class “map” or “object” value type. What looks like structure is usually represented either by dotted hierarchical keys such as general.base_model.0.name, or by arrays. So if you want “nested maps” in an in-memory API, that is a reader-side convenience layer, not an on-disk GGUF primitive. Likewise, tensors are not metadata values; they live in a separate tensor-info table plus the tensor byte region.

Alignment is globally controlled by general.alignment. The spec says it must be a multiple of 8 and defaults to 32 when absent. Padding is applied after the tensor-info table so that tensor_data begins on the next alignment boundary, and each tensor’s relative offset inside tensor_data must also already be aligned. The practical consequence is that you cannot resolve a tensor’s byte position from its descriptor alone; you must compute tensor_data_start = align_up(end_of_tensor_infos, alignment) first.

Version and endianness differences

The current living spec documents three versions. Version 1 was the initial format. Version 2 changed “most countable values” such as lengths to uint64. Version 3 added big-endian support. That means a real-world parser needs at least one branch for older 32-bit count fields if you intend to support v1, and it should definitely support v2 and v3 because public GGUF model repos still contain v2 artifacts.

Endianness deserves extra care. The spec says models are little-endian by default, that big-endian files may exist, and also that “at the time of writing” there is no definitive way to determine big-endianness from the file alone. The reference Python reader works around this by reading the version field and using a heuristic: if the low 16 bits appear zero under host interpretation, it assumes the model was written for the opposite byte order and swaps. That is useful, but it is an implementation heuristic rather than a guaranteed property of the container.

A concrete hex-dump view

The following header fragment is illustrative, not tied to a specific public model file. It shows how to read the first 24 bytes of a little-endian GGUF v3 file. The field meanings come directly from the spec.

00000000  47 47 55 46 03 00 00 00 5b 00 00 00 00 00 00 00
00000010  3c 00 00 00 00 00 00 00

47 47 55 46                         = ASCII "GGUF"
03 00 00 00                         = version 3
5b 00 00 00 00 00 00 00            = tensor_count = 91
3c 00 00 00 00 00 00 00            = metadata_kv_count = 60

A second illustrative fragment shows one metadata entry for general.architecture = "llama". In v2/v3, string lengths are 64-bit. The metadata value type for STRING is 8.

14 00 00 00 00 00 00 00            = key length = 20
67 65 6e 65 72 61 6c 2e 61 72 63 68 69 74 65 63 74 75 72 65
                                     "general.architecture"
08 00 00 00                         = value_type = STRING
05 00 00 00 00 00 00 00            = string length = 5
6c 6c 61 6d 61                      = "llama"

Tooling choices

The tool landscape is surprisingly polarized. If your main job is parsing exact metadata, the best options are either gguf or your own parser. If your main job is running the model, llama.cpp is the canonical executor. If your main job is bringing a GGUF model into a PyTorch workflow, Transformers is the most convenient official integration, but it is materially heavier than a pure binary parser.

A pragmatic split emerges from the sources. Use gguf as the reference parser, use llama.cpp as the behavioral oracle, use Transformers as the high-level compatibility cross-check, and keep your production parser small and self-contained. That is especially appropriate if your long-term objective is replacing a llama.cpp dependency with a dedicated managed-language package rather than wrapping the C++ stack forever.

Reference parsing implementations

The exact parsing sequence

A correct metadata parser for real GGUF LLaMA-family models should proceed in this order.

1. Read the first 4 bytes raw and verify the byte sequence is GGUF.
2. Read the version field, choosing an endianness policy. For v2/v3 support, use 64-bit counts; if you need v1, branch older count widths separately.
3. Read tensor_count and metadata_kv_count.
4. Parse metadata_kv_count entries sequentially:

• read key string,
• read value_type,
• recursively decode the value.

1. Parse tensor_count tensor-info entries sequentially:

• tensor name string,
• dimension count,
• dimensions,
• ggml_type,
• relative tensor offset.

1. Look up general.alignment; if missing, use 32.
2. Compute tensor_data_start by aligning the current file offset upward.
3. Convert each tensor’s relative offset to an absolute file offset with tensor_data_start + rel_offset.
4. Validate metadata and tensor names against the standardized keys and expected LLaMA-family conventions.

Python reference parser

The Python example below is dependency-light and intentionally limited to v2/v3, which is enough for most public GGUF LLaMA-family artifacts in circulation. It parses exact metadata, nested arrays, tensor descriptors, alignment, and tensor byte positions. It does not dequantize tensors, because that is unnecessary for the objective here.

from __future__ import annotations

import io
import mmap
import os
import struct
from dataclasses import dataclass
from enum import IntEnum
from typing import Any, BinaryIO

GGUF_MAGIC = b"GGUF"


class GGUFValueType(IntEnum):
    UINT8 = 0
    INT8 = 1
    UINT16 = 2
    INT16 = 3
    UINT32 = 4
    INT32 = 5
    FLOAT32 = 6
    BOOL = 7
    STRING = 8
    ARRAY = 9
    UINT64 = 10
    INT64 = 11
    FLOAT64 = 12


class GGMLType(IntEnum):
    F32 = 0
    F16 = 1
    Q4_0 = 2
    Q4_1 = 3
    Q5_0 = 6
    Q5_1 = 7
    Q8_0 = 8
    Q8_1 = 9
    Q2_K = 10
    Q3_K = 11
    Q4_K = 12
    Q5_K = 13
    Q6_K = 14
    Q8_K = 15
    IQ2_XXS = 16
    IQ2_XS = 17
    IQ3_XXS = 18
    IQ1_S = 19
    IQ4_NL = 20
    IQ3_S = 21
    IQ2_S = 22
    IQ4_XS = 23
    I8 = 24
    I16 = 25
    I32 = 26
    I64 = 27
    F64 = 28
    IQ1_M = 29
    BF16 = 30
    TQ1_0 = 34
    TQ2_0 = 35
    MXFP4 = 39


@dataclass(frozen=True)
class TensorInfo:
    name: str
    dims: list[int]
    ggml_type: int
    rel_offset: int
    abs_offset: int


@dataclass(frozen=True)
class GGUFFile:
    version: int
    endian: str
    tensor_count: int
    metadata_count: int
    metadata: dict[str, Any]
    tensors: list[TensorInfo]
    tensor_data_start: int
    path: str


class GGUFParseError(Exception):
    """Raised when GGUF parsing fails."""


class Reader:
    """Small helper around a seekable binary stream."""

    def __init__(self, fh: BinaryIO, endian: str):
        self.fh = fh
        self.endian = endian  # "<" or ">"

    def tell(self) -> int:
        return self.fh.tell()

    def seek(self, pos: int) -> None:
        self.fh.seek(pos)

    def read_exact(self, n: int) -> bytes:
        data = self.fh.read(n)
        if len(data) != n:
            raise GGUFParseError(f"Unexpected EOF while reading {n} byte(s)")
        return data

    def unpack(self, fmt: str) -> tuple[Any, ...]:
        size = struct.calcsize(fmt)
        return struct.unpack(self.endian + fmt, self.read_exact(size))

    def u32(self) -> int:
        return self.unpack("I")[0]

    def u64(self) -> int:
        return self.unpack("Q")[0]

    def read_string_v2_v3(self) -> str:
        n = self.u64()
        raw = self.read_exact(n)
        return raw.decode("utf-8")


def align_up(offset: int, alignment: int) -> int:
    if alignment <= 0:
        raise GGUFParseError("Alignment must be positive")
    return offset + ((alignment - (offset % alignment)) % alignment)


def detect_endian_and_version(fh: BinaryIO) -> tuple[str, int]:
    """
    Heuristic only. The GGUF spec explicitly notes that endianness is not
    definitively self-identifying in-file. This routine tries LE first, then BE.
    """
    fh.seek(0)
    magic = fh.read(4)
    if magic != GGUF_MAGIC:
        raise GGUFParseError(f"Bad magic: {magic!r}")

    raw = fh.read(4)
    if len(raw) != 4:
        raise GGUFParseError("Truncated version field")

    v_le = struct.unpack("<I", raw)[0]
    v_be = struct.unpack(">I", raw)[0]

    if v_le in (2, 3):
        return "<", v_le
    if v_be in (2, 3):
        return ">", v_be

    raise GGUFParseError(
        f"Unsupported or ambiguous GGUF version: le={v_le}, be={v_be}. "
        "This concise parser supports v2/v3 only."
    )


def parse_scalar(r: Reader, value_type: GGUFValueType) -> Any:
    if value_type == GGUFValueType.UINT8:
        return r.unpack("B")[0]
    if value_type == GGUFValueType.INT8:
        return r.unpack("b")[0]
    if value_type == GGUFValueType.UINT16:
        return r.unpack("H")[0]
    if value_type == GGUFValueType.INT16:
        return r.unpack("h")[0]
    if value_type == GGUFValueType.UINT32:
        return r.unpack("I")[0]
    if value_type == GGUFValueType.INT32:
        return r.unpack("i")[0]
    if value_type == GGUFValueType.FLOAT32:
        return r.unpack("f")[0]
    if value_type == GGUFValueType.BOOL:
        value = r.unpack("B")[0]
        if value not in (0, 1):
            raise GGUFParseError(f"Invalid GGUF bool byte: {value}")
        return bool(value)
    if value_type == GGUFValueType.UINT64:
        return r.unpack("Q")[0]
    if value_type == GGUFValueType.INT64:
        return r.unpack("q")[0]
    if value_type == GGUFValueType.FLOAT64:
        return r.unpack("d")[0]
    raise GGUFParseError(f"Unsupported scalar type: {value_type}")


def parse_value(r: Reader, value_type: GGUFValueType) -> Any:
    if value_type == GGUFValueType.STRING:
        return r.read_string_v2_v3()

    if value_type == GGUFValueType.ARRAY:
        subtype = GGUFValueType(r.u32())
        length = r.u64()
        return [parse_value(r, subtype) for _ in range(length)]

    return parse_scalar(r, value_type)


def parse_metadata_and_tensors(path: str) -> GGUFFile:
    with open(path, "rb") as fh:
        endian, version = detect_endian_and_version(fh)
        r = Reader(fh, endian)

        fh.seek(8)  # skip magic + version already consumed during detection

        tensor_count = r.u64()
        metadata_count = r.u64()

        metadata: dict[str, Any] = {}
        for _ in range(metadata_count):
            key = r.read_string_v2_v3()
            raw_type = r.u32()
            value_type = GGUFValueType(raw_type)
            metadata[key] = parse_value(r, value_type)

        tensor_headers: list[tuple[str, list[int], int, int]] = []
        for _ in range(tensor_count):
            name = r.read_string_v2_v3()
            n_dims = r.u32()
            dims = [r.u64() for _ in range(n_dims)]
            ggml_type = r.u32()
            rel_offset = r.u64()
            tensor_headers.append((name, dims, ggml_type, rel_offset))

        alignment = int(metadata.get("general.alignment", 32))
        tensor_data_start = align_up(r.tell(), alignment)

        tensors = [
            TensorInfo(
                name=name,
                dims=dims,
                ggml_type=ggml_type,
                rel_offset=rel_offset,
                abs_offset=tensor_data_start + rel_offset,
            )
            for (name, dims, ggml_type, rel_offset) in tensor_headers
        ]

        return GGUFFile(
            version=version,
            endian=endian,
            tensor_count=tensor_count,
            metadata_count=metadata_count,
            metadata=metadata,
            tensors=tensors,
            tensor_data_start=tensor_data_start,
            path=path,
        )


if __name__ == "__main__":
    import json
    import sys

    if len(sys.argv) != 2:
        raise SystemExit(f"Usage: {os.path.basename(sys.argv[0])} <model.gguf>")

    gguf = parse_metadata_and_tensors(sys.argv[1])

    preview = {
        "path": gguf.path,
        "version": gguf.version,
        "endian": "little" if gguf.endian == "<" else "big",
        "tensor_count": gguf.tensor_count,
        "metadata_count": gguf.metadata_count,
        "tensor_data_start": gguf.tensor_data_start,
        "general.architecture": gguf.metadata.get("general.architecture"),
        "general.alignment": gguf.metadata.get("general.alignment", 32),
        "llama.context_length": gguf.metadata.get("llama.context_length"),
        "llama.embedding_length": gguf.metadata.get("llama.embedding_length"),
        "llama.block_count": gguf.metadata.get("llama.block_count"),
        "first_three_tensors": [
            {
                "name": t.name,
                "dims": t.dims,
                "ggml_type": t.ggml_type,
                "abs_offset": t.abs_offset,
            }
            for t in gguf.tensors[:3]
        ],
    }

    print(json.dumps(preview, indent=2, ensure_ascii=False))

That parser mirrors the official sequencing rules from the spec: raw magic check, versioned counts, recursive metadata decode, tensor-info decode, post-descriptor alignment, and tensor absolute offset calculation. It also intentionally stops short of tensor dequantization, which is exactly the right cut line when your immediate goal is metadata parity rather than inference parity. The main deliberate omission is v1 support; adding it requires a branch for older 32-bit count and length fields because the spec says v2 changed most countable values from uint32 to uint64.

C++ reference parser

The C++ example below follows the same idea: no external parser library, recursive metadata decoding, tensor-info parsing, and correct alignment handling. Like the Python version, it is intentionally scoped to v2/v3.

#include <cstdint>
#include <fstream>
#include <iomanip>
#include <iostream>
#include <stdexcept>
#include <string>
#include <unordered_map>
#include <variant>
#include <vector>

enum class GGUFValueType : uint32_t {
    UINT8 = 0,
    INT8 = 1,
    UINT16 = 2,
    INT16 = 3,
    UINT32 = 4,
    INT32 = 5,
    FLOAT32 = 6,
    BOOL = 7,
    STRING = 8,
    ARRAY = 9,
    UINT64 = 10,
    INT64 = 11,
    FLOAT64 = 12
};

struct GGUFValue;
using GGUFArray = std::vector<GGUFValue>;

struct GGUFValue {
    using Variant = std::variant<
        uint8_t, int8_t, uint16_t, int16_t, uint32_t, int32_t,
        float, bool, std::string, GGUFArray, uint64_t, int64_t, double>;
    Variant data;
};

struct TensorInfo {
    std::string name;
    std::vector<uint64_t> dims;
    uint32_t ggml_type;
    uint64_t rel_offset;
    uint64_t abs_offset;
};

struct GGUFFile {
    uint32_t version;
    bool big_endian;
    uint64_t tensor_count;
    uint64_t metadata_count;
    uint32_t alignment;
    std::unordered_map<std::string, GGUFValue> metadata;
    std::vector<TensorInfo> tensors;
    uint64_t tensor_data_start;
};

class ParseError : public std::runtime_error {
public:
    using std::runtime_error::runtime_error;
};

static uint64_t align_up(uint64_t offset, uint32_t alignment) {
    if (alignment == 0) {
        throw ParseError("alignment must be non-zero");
    }
    return offset + ((alignment - (offset % alignment)) % alignment);
}

template <typename T>
T byteswap_if_needed(T value, bool swap);

template <>
uint16_t byteswap_if_needed(uint16_t value, bool swap) {
    if (!swap) return value;
    return static_cast<uint16_t>((value >> 8) | (value << 8));
}

template <>
uint32_t byteswap_if_needed(uint32_t value, bool swap) {
    if (!swap) return value;
    return __builtin_bswap32(value);
}

template <>
uint64_t byteswap_if_needed(uint64_t value, bool swap) {
    if (!swap) return value;
    return __builtin_bswap64(value);
}

class Reader {
public:
    Reader(std::ifstream& in, bool swap_bytes) : in_(in), swap_(swap_bytes) {}

    template <typename T>
    T read_pod() {
        T value{};
        in_.read(reinterpret_cast<char*>(&value), sizeof(T));
        if (!in_) throw ParseError("unexpected EOF");
        return byteswap_if_needed<T>(value, swap_);
    }

    float read_f32() {
        uint32_t bits = read_pod<uint32_t>();
        float out;
        std::memcpy(&out, &bits, sizeof(out));
        return out;
    }

    double read_f64() {
        uint64_t bits = read_pod<uint64_t>();
        double out;
        std::memcpy(&out, &bits, sizeof(out));
        return out;
    }

    std::string read_string_v2_v3() {
        uint64_t len = read_pod<uint64_t>();
        std::string s(len, '\0');
        in_.read(s.data(), static_cast<std::streamsize>(len));
        if (!in_) throw ParseError("unexpected EOF while reading string");
        return s;
    }

    uint64_t tell() {
        return static_cast<uint64_t>(in_.tellg());
    }

private:
    std::ifstream& in_;
    bool swap_;
};

GGUFValue parse_value(Reader& r, GGUFValueType type);

GGUFValue parse_scalar(Reader& r, GGUFValueType type) {
    switch (type) {
        case GGUFValueType::UINT8:  return GGUFValue{r.read_pod<uint8_t>()};
        case GGUFValueType::INT8:   return GGUFValue{r.read_pod<int8_t>()};
        case GGUFValueType::UINT16: return GGUFValue{r.read_pod<uint16_t>()};
        case GGUFValueType::INT16:  return GGUFValue{r.read_pod<int16_t>()};
        case GGUFValueType::UINT32: return GGUFValue{r.read_pod<uint32_t>()};
        case GGUFValueType::INT32:  return GGUFValue{r.read_pod<int32_t>()};
        case GGUFValueType::FLOAT32:return GGUFValue{r.read_f32()};
        case GGUFValueType::BOOL: {
            uint8_t b = r.read_pod<uint8_t>();
            if (b > 1) throw ParseError("invalid GGUF bool");
            return GGUFValue{static_cast<bool>(b)};
        }
        case GGUFValueType::UINT64: return GGUFValue{r.read_pod<uint64_t>()};
        case GGUFValueType::INT64:  return GGUFValue{r.read_pod<int64_t>()};
        case GGUFValueType::FLOAT64:return GGUFValue{r.read_f64()};
        default:
            throw ParseError("unsupported scalar type");
    }
}

GGUFValue parse_value(Reader& r, GGUFValueType type) {
    if (type == GGUFValueType::STRING) {
        return GGUFValue{r.read_string_v2_v3()};
    }

    if (type == GGUFValueType::ARRAY) {
        auto subtype = static_cast<GGUFValueType>(r.read_pod<uint32_t>());
        uint64_t len = r.read_pod<uint64_t>();
        GGUFArray arr;
        arr.reserve(static_cast<size_t>(len));
        for (uint64_t i = 0; i < len; ++i) {
            arr.push_back(parse_value(r, subtype));
        }
        return GGUFValue{std::move(arr)};
    }

    return parse_scalar(r, type);
}

GGUFFile parse_gguf_v2_v3(const std::string& path) {
    std::ifstream in(path, std::ios::binary);
    if (!in) throw ParseError("failed to open file");

    char magic[4];
    in.read(magic, 4);
    if (!in || std::string(magic, 4) != "GGUF") {
        throw ParseError("bad GGUF magic");
    }

    uint32_t raw_version_le = 0;
    in.read(reinterpret_cast<char*>(&raw_version_le), sizeof(raw_version_le));
    if (!in) throw ParseError("truncated version field");

    uint32_t version_le = raw_version_le;
    uint32_t version_be = __builtin_bswap32(raw_version_le);

    bool swap = false;
    uint32_t version = 0;
    if (version_le == 2 || version_le == 3) {
        version = version_le;
        swap = false;
    } else if (version_be == 2 || version_be == 3) {
        version = version_be;
        swap = true;
    } else {
        throw ParseError("unsupported or ambiguous GGUF version");
    }

    Reader r(in, swap);

    GGUFFile out{};
    out.version = version;
    out.big_endian = swap;
    out.tensor_count = r.read_pod<uint64_t>();
    out.metadata_count = r.read_pod<uint64_t>();

    for (uint64_t i = 0; i < out.metadata_count; ++i) {
        std::string key = r.read_string_v2_v3();
        auto type = static_cast<GGUFValueType>(r.read_pod<uint32_t>());
        out.metadata.emplace(std::move(key), parse_value(r, type));
    }

    struct TempTensor {
        std::string name;
        std::vector<uint64_t> dims;
        uint32_t ggml_type;
        uint64_t rel_offset;
    };
    std::vector<TempTensor> temp;
    temp.reserve(static_cast<size_t>(out.tensor_count));

    for (uint64_t i = 0; i < out.tensor_count; ++i) {
        TempTensor t;
        t.name = r.read_string_v2_v3();
        uint32_t n_dims = r.read_pod<uint32_t>();
        t.dims.reserve(n_dims);
        for (uint32_t d = 0; d < n_dims; ++d) {
            t.dims.push_back(r.read_pod<uint64_t>());
        }
        t.ggml_type = r.read_pod<uint32_t>();
        t.rel_offset = r.read_pod<uint64_t>();
        temp.push_back(std::move(t));
    }

    uint32_t alignment = 32;
    if (auto it = out.metadata.find("general.alignment"); it != out.metadata.end()) {
        alignment = std::get<uint32_t>(it->second.data);
    }
    out.alignment = alignment;
    out.tensor_data_start = align_up(r.tell(), alignment);

    out.tensors.reserve(temp.size());
    for (const auto& t : temp) {
        out.tensors.push_back(TensorInfo{
            t.name,
            t.dims,
            t.ggml_type,
            t.rel_offset,
            out.tensor_data_start + t.rel_offset
        });
    }

    return out;
}

The C++ version is appropriate when you want a future loader that can migrate cleanly into a native or managed/native-hybrid runtime. The logic is intentionally the same as the Python reader and the official GGUFReader: recursive metadata decoding, post-table alignment, and tensor offsets resolved relative to the aligned tensor_data base.

Validation and sample public model

Validation is where many “it parses” implementations fail. A GGUF reader should validate both container correctness and LLaMA-family plausibility.

What to validate against the official schema

Start with the generic container and metadata schema.

Then add LLaMA-specific schema checks. For LLaMA-family models, the standardized architecture prefix is llama, so the core metadata should live under keys such as llama.context_length, llama.embedding_length, llama.block_count, llama.feed_forward_length, llama.attention.head_count, llama.attention.head_count_kv when using GQA, llama.attention.layer_norm_rms_epsilon, llama.rope.dimension_count, and llama.rope.freq_base when present. The tokenizer layer should usually expose tokenizer.ggml.model, tokenizer.ggml.tokens, optional scores and token types, and special token IDs. Standardized tensor names should follow the token_embd, output_norm, output, and blk.N.* conventions.

Common real-world variations your parser should tolerate

The spec explicitly allows community namespaced keys, advises readers to support both uint64 and some uint32 metadata values in practice, and notes an older RoPE key, rope.scale_linear, that executors will need to support indefinitely. Public model repos also still expose GGUFv2 artifacts. Those are exactly the cases where brittle binary readers break, even though the file is practically usable.

A second subtlety is that the spec says general.alignment must be a multiple of 8, but the current Python GGUFReader in llama.cpp is stricter in practice and rejects alignments that are not non-zero powers of two. If you are writing a compatibility-focused parser, it is wise to distinguish spec-invalid, implementation-invalid, and accepted-by-reference-reader cases rather than collapsing them together.

Cross-validated sample output for one public GGUF LLaMA-family model

For a concrete public artifact, I recommend using TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF, specifically the file tinyllama-1.1b-chat-v1.0.Q2_K.gguf. The repository page identifies it as a GGUF model with LLaMA architecture, Q2_K quantization, and a file size of 483,116,416 bytes with a published SHA-256 of 030a469a63576d59f601ef5608846b7718eaa884dd820e9aa7493efec1788afa. The underlying TinyLlama model card and project README state that TinyLlama uses the same architecture and tokenizer as Llama 2, with 22 layers, 32 heads, 4 query groups, embedding size 2048, intermediate size 5632, and sequence length 2048. The GGUF repo page also explicitly describes these older artifacts as GGUFv2.

Because the public HTML pages do not expose a raw gguf_dump output inline, the safest way to present a sample is as a cross-validated subset that a correct parser should return for that exact artifact:

file: tinyllama-1.1b-chat-v1.0.Q2_K.gguf
repo: TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF
sha256: 030a469a63576d59f601ef5608846b7718eaa884dd820e9aa7493efec1788afa
gguf_version: 2
general.architecture: "llama"

# LLaMA-family architecture subset expected from metadata
llama.context_length: 2048
llama.embedding_length: 2048
llama.block_count: 22
llama.feed_forward_length: 5632
llama.attention.head_count: 32
llama.attention.head_count_kv: 4

# Artifact facts from the public GGUF release page
quantization: "Q2_K"
size_bytes: 483116416

That subset is rigorous because every row is anchored either in the public GGUF artifact page or in the original TinyLlama model sources. If you want a full exact dump, the reproducible procedure is to download that exact file, verify the SHA-256, and run either the official gguf_dump.py/reader.py utilities or the custom parsers included above.

Operational, legal, and reproducibility considerations

Dependencies, OS support, and memory behavior

For metadata-only parsing, the dependency floor is low. A custom Python parser can use only the standard library; a custom C++ parser can use only the standard library plus ordinary file I/O. If you want to validate against the reference ecosystem, add the gguf Python package. If you want executor-side cross-checks, add llama.cpp. If you want to compare against a PyTorch import path, add Transformers and PyTorch.

The file format itself is not OS-specific. In practice, the official ecosystem supports Windows, macOS, and Linux broadly enough that llama.cpp documents installation routes such as Homebrew, Nix, Winget, Conda, Docker, prebuilt binaries, and source builds, and the Hugging Face GGUF pages repeat Windows, macOS, Linux, and Docker usage examples. For your own parser, standard buffered reads are the safest cross-platform baseline; mmap is attractive because GGUF was explicitly designed with mmap compatibility in mind, but it is not required for metadata extraction.

Memory behavior differs radically by tool. The official GGUFReader uses numpy.memmap, which is convenient and keeps tensor bytes lazily accessible without copying into RAM. By contrast, Hugging Face documents that its GGUF loading path dequantizes the checkpoint to fp32 so the resulting model is fully compatible with PyTorch, which is much heavier than a metadata-only parse. On the TinyLlama GGUF artifact page, the smallest Q2_K file is about 0.48 GB, but the published “Max RAM required” figure is about 2.98 GB with no GPU offload. That difference is exactly why metadata parsing should stop before runtime weight materialization if your immediate goal is container parity.

Legal and ethical considerations

GGUF is only a container. Your legal obligations come from the underlying model license and any additional acceptable use rules, not from GGUF itself. That matters because the license posture across LLaMA-family artifacts varies widely. TinyLlama is published under Apache-2.0 on Hugging Face, while Meta’s Llama 3.1 model card states that Llama 3.1 is released under a custom commercial license, the Llama 3.1 Community License, and explicitly says developers are expected to deploy additional safeguards when building systems with it.

There is also a provenance issue. The GGUF spec includes standardized source and base-model metadata such as general.source.repo_url and general.base_model.* precisely because converted and quantized files often originate from another upstream model. If you redistribute or automate against third-party GGUF files, it is worth validating provenance metadata and upstream license compatibility before publishing derived artifacts.

Reproducibility checklist

A reproducible metadata-parsing experiment for a public LLaMA-family GGUF should pin five things: the exact file, its hash, the parser version, the spec version assumptions, and the validation oracle. For the public example used here, you can pin the GGUF artifact by repository, filename, and SHA-256 from the Hugging Face file page.

A minimal reproducibility recipe is:

1. Download tinyllama-1.1b-chat-v1.0.Q2_K.gguf from TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF.
2. Verify the SHA-256 matches 030a469a63576d59f601ef5608846b7718eaa884dd820e9aa7493efec1788afa.
3. Parse it with either:

• the Python parser above,
• the C++ parser above,
• python /path/to/llama.cpp/gguf-py/examples/reader.py model.gguf,
• or the official gguf_dump.py script.

1. Confirm the subset shown earlier: general.architecture == "llama", GGUF version 2, and TinyLlama’s published Llama-2-compatible architecture values.

If you are building toward a parity-grade managed loader, the cleanest acceptance test is not “did the parser return something,” but “does the parser return the same metadata keys, tensor descriptors, and offsets as the official gguf utilities for the same file.” That gives you a crisp binary milestone before you take on quantized tensor decoding, tokenizer parity, and full forward-pass execution.