Weight file formats

The file formats used to store and share model weights, such as safetensors (common in open-model repositories and GPU serving), GGUF (llama.cpp's single-file format, usually quantized) and older PyTorch pickle files.

Why it matters. The format decides which tools can load a model, and some formats can run code when opened.

How it works

safetensors stores tensors with a simple header and no executable code, and loads quickly; it is the default in many open-model repositories. GGUF is a single-file format for llama.cpp and tools built on it, bundling the weights (often quantized), tokenizer and metadata. PyTorch .bin and .pt files use Python's pickle format, which can execute arbitrary code when loaded.

GPU serving engines such as vLLM and SGLang mainly load safetensors repositories with a configuration file, llama.cpp uses GGUF, and TensorRT-LLM can compile checkpoints into its own engine files. Quantized repositories follow a specific quantization scheme that the engine must support, and converting between formats can change precision.

Prefer safetensors or GGUF files from the maker or another trusted source, and check the license and quantization of a file before comparing its results with others.

Worked example

Illustrative: a 24B-parameter model ships as BF16 safetensors of about 24 x 2 = 48 GB, split into 10 shards of about 4.8 GB. A GGUF at about 4.5 bits per weight is roughly 24 x 4.5 / 8 = 13.5 GB in one file.

Common mistakes

Loading pickle-based weights from an untrusted source, which can run arbitrary code on your machine.
Assuming a GGUF file matches the original safetensors in quality; most GGUFs are quantized.
Converting formats with a mismatched tokenizer or chat template, which breaks outputs without errors.

See it on a real card: Where to run it on Llama 4 Maverick.