The glossary deck
80 terms you'll meet on a model card, one card each. Open a card for a plain explanation, a worked example, the mistakes engineers make, and live values from the model catalog.
- Basics01
Token
The unit models read, write and bill by
Explained - Basics02
Context window
How much a model can read in one request
Explained - Basics03
Max output tokens
The longest reply a model can write at once
Explained - Basics04
Temperature
How random or predictable the output is
Explained - Basics05
Knowledge cutoff
The date the model's training data stops
Explained - Basics06
Latency
How long a request takes to answer
Explained - Pricing07
Input price
Cost per 1M tokens you send to the model
Explained - Pricing08
Output price
Cost per 1M tokens the model generates
Explained - Pricing09
Prompt caching
Discounted reuse of a repeated prompt prefix
Explained - Pricing10
Long-context pricing
Higher rates once a prompt passes a set length
Explained - Pricing11
Batch API
Async bulk requests at a discount
Explained - Pricing12
Flex processing
Cheaper requests with no speed guarantee
Explained - Pricing13
Priority tier
Pay more for steadier speed and availability
Explained - Pricing14
Self-hosting cost
What it costs to run open weights yourself
Explained - Reasoning15
Reasoning tokens
Thinking tokens, billed as output
Explained - Reasoning16
Effort setting
How hard a model thinks before answering
Explained - Reasoning17
Adaptive thinking
The model decides how much to think
Explained - Capabilities18
Multimodal input
Reading images, PDFs, audio or video too
Explained - Capabilities19
Tool use
Letting the model call your functions
Explained - Capabilities20
Structured outputs
Replies constrained to match a JSON schema
Explained - Capabilities21
Computer use
A model operating apps via screen, mouse, keys
Explained - Capabilities22
Embeddings
Numbers that capture meaning, for search
Explained - Capabilities23
Fine-tuning
Training a model further on your own examples
Explained - Inference24
Inference
Running a trained model to get an output
Explained - Inference25
Prefill and decode
Reading the prompt, then writing tokens one by one
Explained - Inference26
Time to first token
Wait before the first token of the answer arrives
Explained - Inference27
Output speed (tokens per second)
How fast output tokens arrive once they start
Explained - Inference28
KV cache
Per-request attention memory that grows with context
Explained - Inference29
Continuous batching
Requests join and leave the batch at every step
Explained - Inference30
Speculative decoding
A fast draft guesses tokens; the big model checks
Explained - Inference31
Streaming
Receiving output as it is generated
Explained - Inference32
Sampling (top-p and top-k)
Rules for picking the next token from probabilities
Explained - Inference33
Log probabilities
The model's confidence in each token it produced
Explained - Inference34
Stop sequences
Text that ends generation when it appears
Explained - Inference35
Serving engine
Software that loads weights and serves requests
Explained - Inference36
GPU memory (VRAM)
Accelerator memory for weights, KV cache and buffers
Explained - Inference37
Tensor and pipeline parallelism
Splitting one model across several GPUs
Explained - Openness38
Open weights
Model parameters you can download and run
Explained - Openness39
Model license
The legal terms for using the model
Explained - Openness40
API model ID
The exact string that selects a model
Explained - Weights & training41
Parameters
The learned numbers that make up a model
Explained - Weights & training42
Weights
The trained parameter values, stored as files
Explained - Weights & training43
Active parameters
Parameters actually used for each token in MoE
Explained - Weights & training44
Mixture of experts
Only part of the model runs for each token
Explained - Weights & training45
Checkpoint
A saved snapshot of a model's weights
Explained - Weights & training46
Precision formats (BF16, FP8, FP4)
How many bits each number in the model uses
Explained - Weights & training47
Quantization
Lower-precision weights to cut memory and cost
Explained - Weights & training48
Weight file formats
How weights are packaged: safetensors, GGUF and more
Explained - Weights & training49
Tokenizer
Splits text into the tokens a model reads
Explained - Weights & training50
Pretraining
Large-scale next-token training on broad data
Explained - Weights & training51
Post-training
Training that turns a base model into an assistant
Explained - Weights & training52
Base vs instruct model
Raw text predictor vs model tuned to follow requests
Explained - Weights & training53
RLHF
Training on human preferences between responses
Explained - Weights & training54
Distillation
Training a small model to imitate a larger one
Explained - Weights & training55
LoRA
Fine-tuning small add-on matrices, not all weights
Explained - Operations56
Rate limits
Caps on requests and tokens per minute
Explained - Operations57
Data residency
Where your prompts are processed and stored
Explained - Operations58
Model deprecation
When a model is retired and stops working
Explained - Operations59
Fast mode
Faster output from the same model, for more
Explained - Evaluation60
Benchmark
A standard test used to score models
Explained - Evaluation61
SWE-bench
Coding benchmark built from real GitHub issues
Explained - Evaluation62
Elo rating
A relative score from head-to-head votes
Explained - Evaluation63
Hallucination
Confident answers that are not true
Explained - Safety64
Model card
Standard document on a model's uses and limits
Explained - Safety65
System card
A maker's release report on capability and risk
Explained - Safety66
Responsible scaling levels (ASL)
Anthropic's safeguard levels tied to capability
Explained - Safety67
Preparedness framework
OpenAI's thresholds for severe model risks
Explained - Safety68
Harmless response rate
Share of harmful requests handled safely
Explained - Safety69
Over-refusal rate
How often benign requests get refused
Explained - Safety70
Prompt injection
Hidden instructions in content the model reads
Explained - Safety71
Refusal fallback
Blocked requests answered by another model
Explained - Safety72
Zero data retention
Provider keeps no copy of prompts or outputs
Explained - Media73
Price per image
What one generated image costs
Explained - Media74
Price per second
What one second of generated video costs
Explained - Media75
Text-to-speech
Turning text into spoken audio
Explained - Media76
Speech-to-text
Transcribing audio into text
Explained - Media77
Word error rate
How many words a transcript gets wrong
Explained - Media78
Diarization
Labelling who spoke when in a recording
Explained - Media79
Content provenance
Records of where media came from
Explained - Media80
Watermarking
Hidden signals that mark AI-generated content
Explained