Elo rating
A relative score from head-to-head comparisons, where people or graders pick the better of two outputs. Differences matter, not absolute values.
Why it matters. Captures quality on open-ended work that has no single right answer.
How it works
An Elo rating is a relative score computed from many pairwise comparisons, where a person or grader picks the better of two outputs for the same prompt. Ratings only mean something relative to other models in the same pool: the bigger the gap, the more often the higher-rated model is expected to win.
Arena-style leaderboards and some makers' internal evaluations use Elo or closely related methods. Ratings shift as new models and votes arrive, and they depend on who votes and what they ask.
Use Elo for open-ended quality where there is no single right answer, such as writing or chat. Don't compare absolute ratings across different leaderboards or across time.
Worked example
Illustrative: with ratings of 1,300 and 1,200, the standard Elo formula expects the higher model to win 1 / (1 + 10^(-100/400)), about 64% of comparisons. With a 20-point gap it is about 53%, close to a coin flip.
Common mistakes
- Comparing ratings from two different leaderboards or evaluation pools.
- Reading a small gap as a clear win when the confidence intervals overlap.
- Assuming votes on general chat prompts reflect your domain, such as code or legal text.
Across the catalog today
Top Elo-style scores
- Claude Opus 5Anthropic1861 Elo
- Claude Fable 5.1Anthropic1853 Elo
- Muse Spark 1.3Meta1754 Elo
- Grok 4.6xAI1753 Elo
- Muse Spark 1.2Meta1631 Elo
- Claude Sonnet 5Anthropic1618 Elo
Self-reported by each maker, from its own card; settings differ between makers.
All 11 GDPval-AA v2 scoresSee it on a real card: Benchmarks on Claude Opus 5.5.