Stop sequences
Strings that end generation as soon as the model produces them. The API stops there and usually leaves the stop sequence out of the output.
Why it matters. They keep output to the part you need and avoid paying for extra output tokens.
How it works
You pass one or more strings, and the server watches the generated text; when one appears, generation stops and the response reports it as the stop reason. Common uses are stopping after one list item, at a closing tag, or before the model starts writing the next turn itself.
Providers cap how many stop sequences you can set, and some reasoning models don't support them. Matching happens on generated text, so a stop sequence can span token boundaries; engines handle this, but how the returned text is trimmed can differ. They are separate from the max output tokens limit and from the end-of-turn token a model emits on its own.
Check the stop reason on every response to tell a stop-sequence end from hitting the token limit or a natural finish. Choose sequences that cannot appear inside a legitimate answer.
Worked example
Illustrative: you need a 120-token answer, but without a stop sequence the model adds about 400 tokens of explanation. At $10 per 1M output tokens over 1M calls, the extra text costs 400 x 1,000,000 / 1,000,000 x $10 = $4,000.
Common mistakes
- Choosing a stop sequence that can occur inside a valid answer, such as a closing brace in nested JSON, which truncates it.
- Not checking the stop reason, so outputs cut off by the token limit look complete.
- Relying on stop sequences with a model that ignores them; check the API docs for each model.
See it on a real card: Build with it on Llama 4 Maverick.