aiaimodel.cards
Weights & training51

Post-training

Training that turns a base model into an assistant

Post-training

Training after pretraining that turns a base model into a usable assistant: supervised fine-tuning on example conversations, then preference or reinforcement learning such as RLHF, plus safety training.

Why it matters. It shapes instruction following, tone, tool use, reasoning style and refusals, which is most of what you notice when using a model.

How it works

Post-training usually starts with supervised fine-tuning on curated prompts and responses. It then steers the model toward preferred behaviour with reinforcement learning from human or AI feedback, preference methods such as DPO, or reinforcement learning on tasks with checkable answers, such as math and code, which is central to reasoning models. Safety training and refusal behaviour are set at this stage too.

It uses much less data than pretraining but has an outsized effect on behaviour. Makers describe it in system and model cards in varying detail, and a provider can release an updated snapshot with new post-training on the same base.

Differences between models of similar raw capability, such as verbosity, format adherence or over-refusal, often come from post-training. Your own fine-tuning is a small extra post-training step, and it can weaken safety behaviour, which is one reason providers restrict or review it.

Worked example

Illustrative: 1 million supervised examples averaging 1,500 tokens is 1.5 billion tokens. Against a 15-trillion-token pretraining corpus, that is 1.5 x 10^9 / 1.5 x 10^13 = 0.01% of the pretraining tokens.

Common mistakes

  • Attributing differences in verbosity or refusals to model size when they come from post-training.
  • Assuming your fine-tune keeps the original safety behaviour; evaluate it again.
  • Reading "same base model" as "same behaviour" when post-training differs between checkpoints.

Related terms