Every byte and number in this article comes from a single real file. All terminal output comes from one unedited recorded session. The test environment is listed at the end.
There is a file on disk with an unremarkable name:
gemma-4-12B-it-QAT-Q4_0.ggufIt takes up almost seven gigabytes and knows how to talk.
LM Studio opens it without a separate configuration file. llama.cpp understands the same file without a directory of supplementary descriptions, a database, or a neighboring JSON file. Everything it needs—the model architecture, parameters, tokenizer, chat template, and the weights themselves—is stored inside.
That is how GGUF works: it is not merely an archive of tensors, but a self-describing container. A loader does not need to know in advance what the file contains. It can open the file, read the header, and then proceed through a structure described by the file itself.
Let us see just how literally we should interpret the phrase “self-describing.”
We will start at byte zero and sequentially read the header, metadata, and tensor directory, without opening the weights themselves. Then we will try to predict the exact size of the seven-gigabyte file from its structure alone.
If the result matches stat down to the last byte, then the file really has told us everything about itself.
We will use the public model gemma-4-12B-it-QAT-Q4_0.gguf. The file is 6.5 GiB, supports resumable downloads, and is saved directly to the current directory:
$ curl -LO https://huggingface.co/lmstudio-community/\
gemma-4-12B-it-QAT-GGUF/resolve/main/gemma-4-12B-it-QAT-Q4_0.ggufNow we can begin examining it.
The First 24 Bytes
A binary format is best studied not from a description, but from real bytes. Let us begin by printing the first 32:
$ xxd -l 32 gemma-4-12B-it-QAT-Q4_0.gguf
00000000: 4747 5546 0300 0000 9b02 0000 0000 0000 GGUF............
00000010: 2d00 0000 0000 0000 1400 0000 0000 0000 -...............At first glance, this is just a sequence of hexadecimal numbers. But the structure begins to reveal itself immediately.
The first four bytes are:
47 47 55 46In ASCII, this is the string GGUF—the format’s so-called magic number. A magic value lets the loader quickly determine what kind of file it has opened, even before parsing the rest of the contents.
The next four bytes are:
03 00 00 00This is a 32-bit integer with the value 3, meaning that the format version is GGUF v3.
Numbers in GGUF are stored in little-endian order: the least significant byte comes first, followed by the more significant bytes. Therefore, the sequence
9b 02 00 00 00 00 00 00must be read as the 64-bit number 0x29B, or 667 in decimal. That is exactly how many tensors the file contains.
The next eight bytes are:
2d 00 00 00 00 00 00 00They give the number 45—the number of metadata key-value pairs.
The entire mandatory header fits into 24 bytes:
- four bytes of magic;
- four bytes for the version;
- eight bytes for the tensor count;
- eight bytes for the metadata entry count.
The next eight bytes are no longer part of the header:
14 00 00 00 00 00 00 00This is the number 20—the length of the first metadata string.
Immediately after the compact header, the file begins introducing itself. The first 20-character string is the key:
general.architectureMetadata: The Model’s Passport
The header is followed by 45 key-value pairs.
A key is always represented as a length-prefixed string. A value may have one of thirteen types: eight integer types, two floating-point types, bool, string, or array.
In the parser, those wire ids are not bare integers — they map onto an enum (strum derives the id-to-variant conversion and the printed name), so an unknown id fails loudly at the boundary instead of traveling through the code:
/// The 13 GGUF metadata value types, by their wire ids.
#[derive(Debug, Clone, Copy, PartialEq, Eq, FromRepr, Display)]
#[strum(serialize_all = "lowercase")]
#[repr(u32)]
pub enum ValueType {
U8 = 0, I8 = 1, U16 = 2, I16 = 3, U32 = 4, I32 = 5, F32 = 6,
Bool = 7, String = 8, Array = 9, U64 = 10, I64 = 11, F64 = 12,
}Each parsed value lands in a typed MetaValue enum with the same thirteen variants; an array includes both its element type and its number of values, so the structure can describe anything from small lists of numbers to hundreds of thousands of tokenizer strings.
A small Rust parser walks through the entries sequentially and prints the file’s contents (its full source is in the appendix at the end):
$ gguf-info gemma-4-12B-it-QAT-Q4_0.gguf
header: GGUF v3, 667 tensors, 45 metadata pairs
metadata (38 scalars, 7 arrays, ends at byte 15782884):
general.architecture gemma4
general.size_label 12B
gemma4.block_count 48
gemma4.embedding_length 3840
gemma4.feed_forward_length 15360
gemma4.attention.head_count 16
gemma4.context_length 262144
gemma4.attention.sliding_window 1024
general.sampling.top_k 64
general.sampling.top_p 0.95
general.sampling.temp 1
tokenizer.chat_template {%- macro format_type_argument(type_value) -%}…
…
tokenizer.ggml.tokens [string; 262144]
tokenizer.ggml.scores [f32; 262144]
tokenizer.ggml.token_type [i32; 262144]
tokenizer.ggml.merges [string; 514906]At this point, we already know most of what would normally have to be looked up in a separate configuration file.
The model reports the architecture name—gemma4; the number of transformer blocks—48; the hidden representation size—3,840; the feed-forward layer size—15,360; the number of attention heads—16; and the maximum context length—262,144 tokens.
To clarify: the architecture name determines how the model is structured; the number of transformer blocks indicates its depth; the hidden representation size is the amount of data passed between layers; the feed-forward layer size is the width of the internal transformations; the number of attention heads is the number of parallel ways the model can analyze context; and the maximum context is the amount of text the model can consider at once.
But the metadata is not limited to the architecture.
It also contains the recommended sampling values:
top_k = 64
top_p = 0.95
temperature = 1The file also includes a Jinja chat template and the IDs of special-purpose tokens. As a result, the runtime receives not only a set of matrices, but also information about how to prepare a conversation and interpret the model’s output.
“One file” is not a marketing simplification here. It is one of the format’s defining properties.
Why the Metadata Takes Up Almost 16 Megabytes
A binary file header is usually expected to be small: a few numbers, flags, and strings. In our case, however, the metadata section ends at byte:
15 782 884That is almost sixteen megabytes before the tensor directory and the weights themselves even begin.
Most of this space is not used to describe the architecture. Numbers such as the layer count or context length occupy only a handful of bytes.
The main consumer is the tokenizer:
tokenizer.ggml.tokens [string; 262144]
tokenizer.ggml.scores [f32; 262144]
tokenizer.ggml.token_type [i32; 262144]
tokenizer.ggml.merges [string; 514906]The file contains 262,144 vocabulary strings, the same number of token scores and token types, and 514,906 BPE merges.
A BPE merge is a rule that tells the tokenizer to combine two adjacent tokens into a larger one; the list of 514,906 merges defines the order and priority of those combinations when text is tokenized.
The model stores not only the ability to continue a sequence of tokens, but also the rules by which the original text is converted into those tokens.
The largest part of its “profile” is not the architecture parameters, but the language vocabulary.
Lengths Read from a File Cannot Be Trusted
To read a string, the parser first obtains its length and then consumes the corresponding number of bytes:
pub fn string(&mut self) -> Result<String> {
let len = self.u64()?;
ensure!(len < 1 << 20, "string length {len} is hostile");
Ok(String::from_utf8_lossy(&self.bytes(len as usize)?).into_owned())
}The check
ensure!(len < 1 << 20, ...)is not part of GGUF’s logic. It protects the parser itself.
It is important to understand that every length stored inside a binary file is a value supplied by an external source. A corrupted or deliberately crafted file may claim that the next string is several gigabytes long. If the loader trusts that number unconditionally, it may try to allocate an enormous buffer, read beyond the end of the file, or overflow while calculating offsets.
That is why the length is validated before the data is read.
A string that claims to be two gigabytes long is not necessarily a large string. It may be an attack.
In this article, we are simply reading a valid file carefully. But fields like these are exactly what loader fuzzing usually targets: random and boundary values are substituted to find places where a parser trusts its input too much.
The Tensor Directory
The tensor directory begins immediately after the metadata.
It does not contain the weights themselves. It is a table of contents in which each tensor has the following fields:
- name;
- number of dimensions;
- dimensions;
- storage type;
- offset within the data section.
The directory contains 667 entries in total. On average, each one occupies about sixty bytes:
tensor directory (ends at byte 15822741):
output_norm.weight 3840 F32 offset 0
rope_freqs.weight 256 F32 offset 15360
token_embd.weight 3840 × 262144 Q6_K offset 16384
blk.0.attn_k.weight 3840 × 2048 Q4_0 offset 825769984
… 659 more …
blk.47.ffn_up.weight 3840 × 15360 Q4_0 offset 6926847456
blk.47.post_ffw_norm.weight 3840 F32 offset 6960040448The tensor names follow a regular structure.
There are global tensors:
output_norm.weight
rope_freqs.weight
token_embd.weightThey are followed by blocks:
blk.0
blk.1
…
blk.47Each block contains attention matrices, feed-forward network matrices, and normalization tensors.
The metadata has already told us that the architecture contains 48 blocks. The directory confirms this not as a declaration, but through concrete addresses ranging from blk.0 to blk.47.
Offsets are measured not from the beginning of the file, but from the beginning of the data section. This matters: the loader locates the start of the weights once, then obtains the address of any tensor with a simple addition:
tensor_address = data_start + tensor_offsetThe Model Can Be Counted Without Reading the Weights
The directory contains the dimensions of every tensor. This means the number of parameters in a tensor can be obtained with ordinary multiplication.
For the matrix:
3840 × 2048that gives:
7 864 320 parametersNow sum the products of the dimensions for all 667 tensors:
parameters: 11.91 B (11907350576)
weights: 6.48 GiB
quantization mix:
F32 338 tensors 0.00 GiB 0.0%
Q4_0 328 tensors 5.71 GiB 88.1%
Q6_K 1 tensors 0.77 GiB 11.9%The result is:
11 907 350 576 parametersThis is where the 12B label comes from. It is not a separate number that must be taken on trust: it can be reconstructed from the directory.
The quantization mix is visible there as well.
Of the 667 tensors:
- 338 are stored in F32;
- 328 are stored in Q4_0;
- one is stored in Q6_K.
The tensor count alone says little about how much space they occupy. Hundreds of F32 tensors may be almost negligible if each contains only a few thousand elements. A single enormous matrix, meanwhile, can account for a substantial portion of the file.
That is exactly what happens here.
What Q4_0 Means
Without quantization, each F32 weight occupies four bytes, or 32 bits.
Quantization stores model weights in a more compact format using fewer bits, reducing the model’s size and potentially speeding up computation. Instead of
F32, formats such asF16,INT8, orINT4may be used, as well as more complex formats that are not aligned to byte boundaries.
For a model with 11.9 billion parameters, F32 would require tens of gigabytes for the weights alone. To reduce the size, the values are stored in quantized form.
In Q4_0, weights are divided into blocks of 32 values. One such block occupies 18 bytes:
- 16 bytes contain 32 four-bit values;
- another two bytes store a shared scale for the block.
On average, this gives:
18 × 8 / 32 = 4.5 bits per weightThere is no separate “half-bit” attached to each weight: the block can be represented as a structure such as
{ scale: f16, values: [u8; 16] }. The scale’s 16 bits are shared across 32 weights, so each weight accounts for an average of another16 / 32 = 0.5bits on top of the four bits used for the value itself.
This is how the 328 main attention and feed-forward matrices are stored. Together they occupy 5.71 GiB, or 88.1% of all weight data.
But one tensor breaks the pattern:
token_embd.weight 3840 × 262144 Q6_KThis is the vocabulary embedding matrix. It contains more than a billion parameters—about 8.5% of the entire model—and occupies 0.77 GiB.
Embeddings are numerical vectors through which the model represents tokens and their meaning. A researcher usually cannot say what any individual number means: the meaning is distributed across many coordinates at once.
This tensor uses Q6_K quantization, with an average density of 6.56 bits per weight.
In other words, the quantizer allocated roughly one and a half times as many bits per parameter to the embeddings as it did to most of the other matrices.
The reason lies in the importance of this tensor: the embedding participates in the representation of every input token and is tied to the model’s output vocabulary. An error in it affects not just one layer or block, but the entire stream of tokens.
The directory does not explain why the quantizer made this decision. But it does reveal the decision itself: almost the entire model is stored in Q4_0, while the vocabulary receives the more precise Q6_K format.
The remaining 338 F32 tensors are mostly small normalization tensors. They are numerous, but their combined size does not even register at the hundredth-of-a-percent level.
Saving a few bits on tiny tensors is almost pointless, so they can remain at full precision without noticeably increasing the file size.
Predicting the File Size
By this point, we have read the header, metadata, and tensor directory.
The parser has reached byte:
15 822 741But the weights themselves, which occupy 6.48 GiB, have not been opened.
Now let us see whether the information already obtained is enough to calculate the size of the entire file.
The data section must begin at an aligned address. The required alignment is stored in the metadata and is 32 bytes.
The directory ends at offset:
15 822 741The next address divisible by 32 is:
15 822 752There are eleven bytes of padding between the directory and the data.
Padding carries no useful values. It ensures that the data section begins on a boundary convenient for memory access.
The directory also reports the offset and type of every tensor. Given its dimensions and quantization format, we can calculate how many bytes it occupies.
The farthest tensor determines the end of the data section:
let data_start = directory_end.div_ceil(alignment) * alignment;
let predicted = data_start + data_size;The calculation produces:
predicted: 6975878560 bytes
actual: 6975878560 bytes (stat)
EXACTPredicted size:
6 975 878 560 bytesActual size:
6 975 878 560 bytesThe match is exact.
We did not read a single weight, yet we were able to state the size of a seven-gigabyte file down to the final byte.
GGUF’s self-describing nature is literal here: the file contains enough information to reconstruct not only the names and shapes of its tensors, but also its own physical layout.
Why Offsets Are Relative to the Data Section
The absolute position of each tensor could have been stored relative to the beginning of the file. GGUF instead uses offsets relative to the beginning of the data section.
This divides the file into two logical parts:
- the description;
- the weights themselves.
The loader first parses the relatively small region containing the header, metadata, and directory. It then calculates data_start and interprets every offset relative to that point.
This approach is convenient for memory mapping.
The operating system can create a memory mapping, after which the file’s contents become available to the process as a range of virtual memory. To access a particular tensor, there is no need to read all preceding data sequentially or unpack a shared archive.
It is enough to calculate:
data_start + offsetand take a slice of the required length.
Of course, the physical file pages still have to be brought into memory as they are accessed. But the operating system handles that. The runtime does not need to copy all 6.5 GiB into a separate buffer in advance.
This is why large model files can open quickly: “loading a model” does not always mean immediately reading every byte of its weights.
How a model larger than the available RAM operates through memory mapping and page eviction is a separate topic.
One File Instead of a Set of Conventions
GGUF solves several problems at once.
The metadata describes the architecture and runtime parameters. Tokenizer arrays make it possible to convert text into tokens without an external vocabulary. The directory maps tensor names to their shapes, types, and addresses. The data section stores the weights themselves in formats the runtime understands.
As a result, the same file can be passed to different compatible programs. They do not need to agree on the name of a neighboring JSON file, the directory layout, or how separate binary fragments should be matched together.
The format becomes the contract.
This also makes independent analysis tools possible. Our gguf-info does not run the model, implement a transformer, or know how to generate text. It consists of roughly four hundred lines of Rust — a module that types the wire format and a binary that reads and reports — sequentially walking the file’s self-description.
Such a tool can answer practical questions before the weights are loaded:
- which architecture is stored inside;
- how many layers it has;
- what context size it declares;
- which tokenizer it uses;
- how many parameters the model contains;
- which quantization formats are used;
- how much space each tensor occupies;
- where the data section begins and ends.
In effect, it is an equivalent of file(1) for models.
Self-Description Makes the Format Verifiable
A self-describing format has another advantage: its claims can be cross-checked.
The header declares 667 tensors, so the parser must read exactly 667 directory entries.
The metadata declares 48 blocks, so the names should include blk.0 through blk.47.
The 12B label can be compared with the sum of the products of the tensor dimensions:
11 907 350 576The tensor offsets, types, and shapes make it possible to calculate the total volume of the weights.
The alignment and the end of the directory determine the start of the data section.
The last tensor determines the end of the file.
Any of these relationships can serve as an integrity check. If the numbers stop agreeing, the file is corrupted, truncated, or incorrectly constructed.
As mentioned earlier, this does not mean that GGUF is automatically safe from every kind of invalid input. A parser must still validate lengths, arithmetic overflows, offset ranges, and the file’s actual size.
But the format provides enough structure for those checks to be possible in the first place.
What Is Inside a Model File
From the outside, GGUF looks like one large binary object. Inside, its organization is much easier to understand:
- a 24-byte header reports the version and the number of records that follow;
- metadata describes the architecture, tokenizer, and runtime parameters;
- the directory lists the tensors, their dimensions, types, and offsets;
- an aligned data section contains the quantized weights.
In our file, the description ends at approximately the sixteenth megabyte. It is followed by 6.48 GiB of weights for an 11.9-billion-parameter model.
Most of the working matrices are packed in Q4_0. The enormous vocabulary embedding is stored at higher precision in Q6_K. Small normalization tensors remain in F32.
We were able to determine all of this without running the model or reading the contents of its tensors.
Finally, the file passed the strictest test: from its description alone, we calculated its actual size—6 975 878 560 bytes.
A match down to the byte is the best possible answer to the question of what a self-describing format means.
GGUF does not merely store weights.
It explains what those weights are, how they are laid out, and where the file itself ends.
Appendix: Full Source Files
format.rs — the wire format as types
//! The GGUF wire format as Rust types: every id the file can utter is an
//! enum variant, every value a typed `MetaValue` — no stringly-typed
//! parsing, no bare integers. Reading stays explicit byte-walking on
//! purpose: the format is the subject here, and a serde Deserializer
//! would wrap these exact reads in trait plumbing without removing one.
//!
//! Every length that arrives from the file is untrusted and checked
//! before use — a taste of the discipline the fuzzing episode expands.
use anyhow::{Context, Result, ensure};
use std::fmt;
use std::fs::File;
use std::io::{BufReader, Read};
use strum::{Display, FromRepr};
/// Sequential reader that tracks its absolute offset in the file.
pub struct Cursor {
reader: BufReader<File>,
pub position: u64,
}
impl Cursor {
pub fn new(file: File) -> Self {
Self {
reader: BufReader::new(file),
position: 0,
}
}
pub fn bytes(&mut self, len: usize) -> Result<Vec<u8>> {
let mut buffer = vec![0_u8; len];
self.reader
.read_exact(&mut buffer)
.with_context(|| format!("unexpected end of file at offset {}", self.position))?;
self.position += len as u64;
Ok(buffer)
}
pub fn u32(&mut self) -> Result<u32> {
Ok(u32::from_le_bytes(self.bytes(4)?.try_into().unwrap()))
}
pub fn u64(&mut self) -> Result<u64> {
Ok(u64::from_le_bytes(self.bytes(8)?.try_into().unwrap()))
}
pub fn string(&mut self) -> Result<String> {
let len = self.u64()?;
ensure!(len < 1 << 20, "string length {len} is hostile");
Ok(String::from_utf8_lossy(&self.bytes(len as usize)?).into_owned())
}
}
/// The 13 GGUF metadata value types, by their wire ids.
#[derive(Debug, Clone, Copy, PartialEq, Eq, FromRepr, Display)]
#[strum(serialize_all = "lowercase")]
#[repr(u32)]
pub enum ValueType {
U8 = 0,
I8 = 1,
U16 = 2,
I16 = 3,
U32 = 4,
I32 = 5,
F32 = 6,
Bool = 7,
String = 8,
Array = 9,
U64 = 10,
I64 = 11,
F64 = 12,
}
impl ValueType {
pub fn parse(id: u32) -> Result<Self> {
Self::from_repr(id).with_context(|| format!("unknown metadata value type {id}"))
}
}
/// One parsed metadata value — arrays keep their shape, not their body.
pub enum MetaValue {
U8(u8),
I8(i8),
U16(u16),
I16(i16),
U32(u32),
I32(i32),
F32(f32),
Bool(bool),
String(String),
Array { elem: ValueType, count: u64 },
U64(u64),
I64(i64),
F64(f64),
}
impl MetaValue {
pub fn read(cursor: &mut Cursor, kind: ValueType) -> Result<Self> {
Ok(match kind {
ValueType::U8 => Self::U8(cursor.bytes(1)?[0]),
ValueType::I8 => Self::I8(cursor.bytes(1)?[0] as i8),
ValueType::U16 => Self::U16(u16::from_le_bytes(cursor.bytes(2)?.try_into().unwrap())),
ValueType::I16 => Self::I16(i16::from_le_bytes(cursor.bytes(2)?.try_into().unwrap())),
ValueType::U32 => Self::U32(cursor.u32()?),
ValueType::I32 => Self::I32(cursor.u32()? as i32),
ValueType::F32 => Self::F32(f32::from_le_bytes(cursor.bytes(4)?.try_into().unwrap())),
ValueType::Bool => Self::Bool(cursor.bytes(1)?[0] != 0),
ValueType::String => Self::String(cursor.string()?),
ValueType::Array => {
let elem = ValueType::parse(cursor.u32()?)?;
let count = cursor.u64()?;
ensure!(count < 1 << 32, "array length {count} is hostile");
for _ in 0..count {
Self::read(cursor, elem)?;
}
Self::Array { elem, count }
}
ValueType::U64 => Self::U64(cursor.u64()?),
ValueType::I64 => Self::I64(cursor.u64()? as i64),
ValueType::F64 => Self::F64(f64::from_le_bytes(cursor.bytes(8)?.try_into().unwrap())),
})
}
/// Integer view, for values that configure the parser (alignment).
pub fn as_u64(&self) -> Option<u64> {
match self {
Self::U8(value) => Some(*value as u64),
Self::U16(value) => Some(*value as u64),
Self::U32(value) => Some(*value as u64),
Self::U64(value) => Some(*value),
_ => None,
}
}
}
impl fmt::Display for MetaValue {
fn fmt(&self, out: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::U8(value) => write!(out, "{value}"),
Self::I8(value) => write!(out, "{value}"),
Self::U16(value) => write!(out, "{value}"),
Self::I16(value) => write!(out, "{value}"),
Self::U32(value) => write!(out, "{value}"),
Self::I32(value) => write!(out, "{value}"),
Self::F32(value) => write!(out, "{value}"),
Self::Bool(value) => write!(out, "{value}"),
Self::String(value) => write!(out, "{value}"),
Self::Array { elem, count } => write!(out, "[{elem}; {count}]"),
Self::U64(value) => write!(out, "{value}"),
Self::I64(value) => write!(out, "{value}"),
Self::F64(value) => write!(out, "{value}"),
}
}
}
/// The ggml tensor types this file may carry, by their wire ids.
/// Unknown ids fail loudly instead of guessing.
#[allow(non_camel_case_types)] // wire names: Q4_0 is the type's real name
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, FromRepr, Display)]
#[repr(u32)]
pub enum GgmlType {
F32 = 0,
F16 = 1,
Q4_0 = 2,
Q4_1 = 3,
Q5_0 = 6,
Q5_1 = 7,
Q8_0 = 8,
Q2_K = 10,
Q3_K = 11,
Q4_K = 12,
Q5_K = 13,
Q6_K = 14,
BF16 = 30,
}
impl GgmlType {
pub fn parse(id: u32) -> Result<Self> {
Self::from_repr(id).with_context(|| format!("unknown tensor type {id}"))
}
/// (bytes per block, elements per block).
pub fn block(self) -> (u64, u64) {
match self {
Self::F32 => (4, 1),
Self::F16 | Self::BF16 => (2, 1),
Self::Q4_0 => (18, 32),
Self::Q4_1 => (20, 32),
Self::Q5_0 => (22, 32),
Self::Q5_1 => (24, 32),
Self::Q8_0 => (34, 32),
Self::Q2_K => (84, 256),
Self::Q3_K => (110, 256),
Self::Q4_K => (144, 256),
Self::Q5_K => (176, 256),
Self::Q6_K => (210, 256),
}
}
}
/// One tensor directory entry.
pub struct Tensor {
pub name: String,
pub shape: Vec<u64>,
pub kind: GgmlType,
pub offset: u64,
}
impl Tensor {
pub fn read(cursor: &mut Cursor) -> Result<Self> {
let name = cursor.string()?;
let dims = cursor.u32()?;
ensure!(dims <= 4, "tensor {name} claims {dims} dimensions");
let shape: Vec<u64> = (0..dims).map(|_| cursor.u64()).collect::<Result<_>>()?;
let kind = GgmlType::parse(cursor.u32()?)?;
let offset = cursor.u64()?;
Ok(Self {
name,
shape,
kind,
offset,
})
}
pub fn elements(&self) -> u64 {
self.shape.iter().product()
}
pub fn byte_size(&self) -> Result<u64> {
let (block_bytes, block_elems) = self.kind.block();
let elements = self.elements();
ensure!(
elements % block_elems == 0,
"tensor {} has {} elements, not divisible into {}-element blocks",
self.name,
elements,
block_elems
);
Ok(elements / block_elems * block_bytes)
}
}main.rs — the gguf-info report
//! Scene 02: `file(1)` for model files — read what a GGUF says about itself.
//!
//! The program walks the three layers of the container in order — header,
//! key–value metadata, tensor directory — WITHOUT touching tensor data.
//! Everything it prints is read or computed from those bytes alone,
//! including the exact file size the directory implies. The format is
//! self-describing; this program is the proof.
mod format;
use anyhow::{Context, Result, ensure};
use clap::Parser;
use format::{Cursor, GgmlType, MetaValue, Tensor, ValueType};
use std::collections::BTreeMap;
use std::fs::File;
use std::path::PathBuf;
/// Prints architecture, metadata, tensor table, quantization mix and the
/// predicted file size of a .gguf — from its self-description alone.
#[derive(Parser)]
struct Args {
/// Path to the .gguf file
model: PathBuf,
/// How many tensor-table rows to print (head and tail)
#[arg(long, default_value_t = 4)]
rows: usize,
}
fn human(bytes: u64) -> String {
format!("{:.2} GiB", bytes as f64 / (1 << 30) as f64)
}
fn main() -> Result<()> {
let args = Args::parse();
let file = File::open(&args.model).with_context(|| format!("open {:?}", args.model))?;
let actual_size = file.metadata()?.len();
let mut cursor = Cursor::new(file);
// Layer 1: the 24-byte header.
let magic = cursor.bytes(4)?;
ensure!(&magic == b"GGUF", "not a GGUF file (magic {magic:02x?})");
let version = cursor.u32()?;
ensure!(version == 3, "unsupported GGUF version {version}");
let tensor_count = cursor.u64()?;
ensure!(
tensor_count < 100_000,
"tensor count {tensor_count} is hostile"
);
let kv_count = cursor.u64()?;
ensure!(kv_count < 10_000, "metadata count {kv_count} is hostile");
println!("header: GGUF v{version}, {tensor_count} tensors, {kv_count} metadata pairs");
// Layer 2: key–value metadata. Scalars are kept, arrays are walked
// (their bytes must be consumed to find the tensor directory) but
// only their shape is remembered.
let mut alignment = 32_u64;
let mut scalars: Vec<(String, MetaValue)> = Vec::new();
let mut arrays: Vec<(String, MetaValue)> = Vec::new();
for _ in 0..kv_count {
let key = cursor.string()?;
let kind = ValueType::parse(cursor.u32()?)?;
let value = MetaValue::read(&mut cursor, kind)?;
if key == "general.alignment" {
alignment = value
.as_u64()
.with_context(|| format!("general.alignment is not an integer ({value})"))?;
}
match value {
MetaValue::Array { .. } => arrays.push((key, value)),
scalar => scalars.push((key, scalar)),
}
}
let metadata_end = cursor.position;
println!(
"\nmetadata ({} scalars, {} arrays, ends at byte {metadata_end}):",
scalars.len(),
arrays.len()
);
for (key, value) in scalars.iter().chain(&arrays) {
let mut text = value.to_string();
if text.len() > 50 {
text.truncate(47);
text.push('…');
}
println!(" {key:<44} {text}");
}
// Layer 3: the tensor directory.
let tensors: Vec<Tensor> = (0..tensor_count)
.map(|_| Tensor::read(&mut cursor))
.collect::<Result<_>>()?;
let directory_end = cursor.position;
println!("\ntensor directory (ends at byte {directory_end}):");
let show: Vec<usize> = (0..tensors.len().min(args.rows))
.chain(tensors.len().saturating_sub(args.rows)..tensors.len())
.collect();
let mut last_shown = None;
for index in show {
if last_shown == Some(index) {
continue;
}
if last_shown.is_some_and(|last| index != last + 1) {
println!(" … {} more …", tensors.len() - 2 * args.rows);
}
let tensor = &tensors[index];
let shape = tensor
.shape
.iter()
.map(u64::to_string)
.collect::<Vec<_>>()
.join(" × ");
println!(
" {:<28} {:<16} {:<5} offset {:>12}",
tensor.name, shape, tensor.kind, tensor.offset
);
last_shown = Some(index);
}
// What the directory implies: parameter count, quantization mix, and
// the exact size of the file that must follow.
let mut by_type: BTreeMap<GgmlType, (u64, u64)> = BTreeMap::new();
let mut parameters = 0_u64;
let mut data_size = 0_u64;
for tensor in &tensors {
let size = tensor.byte_size()?;
let entry = by_type.entry(tensor.kind).or_default();
entry.0 += 1;
entry.1 += size;
parameters += tensor.elements();
data_size = data_size.max(tensor.offset + size);
}
println!(
"\nparameters: {:.2} B ({parameters})",
parameters as f64 / 1e9
);
println!("weights: {}", human(data_size));
println!("quantization mix:");
for (kind, (count, bytes)) in &by_type {
let share = *bytes as f64 / data_size as f64 * 100.0;
println!(
" {:<5} {count:>4} tensors {:>10} {share:4.1}%",
kind.to_string(),
human(*bytes)
);
}
// The self-description test: header + metadata + directory + padding
// + data must land exactly on the file's last byte.
let data_start = directory_end.div_ceil(alignment) * alignment;
let predicted = data_start + data_size;
println!("\nalignment: {alignment} (data begins at byte {data_start})");
println!("predicted: {predicted} bytes");
println!("actual: {actual_size} bytes (stat)");
println!(
"{}",
if predicted == actual_size {
"EXACT"
} else {
"MISMATCH"
}
);
Ok(())
}Test environment: gemma-4-12B-it-QAT-Q4_0.gguf (lmstudio-community), 6 975 878 560 bytes, GGUF v3; parser: approximately 400 lines of Rust 1.97.1 (a wire-format module and a report binary), release build; tensor data was not read; Fedora 43. All terminal blocks come from one unedited recorded session.