aiaimodel.cards
← All benchmarks

MMLU Pro

Multiple-choice questions across dozens of academic subjects.

Your workloadRAG over docs · 12K in · 600 out · 50K requests/mo · 30% cached
  1. 1Seed 2.0 Lite · ByteDancebest value
    87.7%
  2. 2Seed 2.0 Pro · ByteDance
    87.0%
  3. 3Gemma 4 31B · Googlebest value
    85.2%
  4. 4Llama 4 Maverick · Meta
    80.5%
  5. 5Llama 4 Scout · Metabest value
    74.3%

“Best value”: no other model here reports a higher score at a lower cost for your workload. Scores are self-reported by each maker with different settings, so treat close calls as ties.

How each maker reported it

  • Seed 2.0 Lite Tables 4, 8, 12 (pp.10, 13, 17)

    ByteDance's own runs from the Seed2.0 model card, which predates the 260428 snapshot served now. Codeforces is an Elo rating. Seed 2.0 Mini is shown for comparison (same table).

  • Seed 2.0 Pro Tables 3, 8, 11 (pp.9, 13, 16)

    ByteDance's own runs. Competitor columns are as printed in the Seed2.0 card (frontier models available in early 2026). Codeforces is an Elo rating. HLE-text under Agentic is with search tools.

  • Gemma 4 31B Benchmark results

    Instruction-tuned models, as published in Google's model card. OmniDocBench is edit distance (lower is better). Audio benchmarks are omitted because 31B has no audio input.

  • Llama 4 Maverick Benchmarks: Instruction tuned models

    Meta's own 0-shot results on bf16 weights. Llama 3.x models have no image support and a 128K context, so those cells are not reported.

  • Llama 4 Scout Benchmarks: Instruction tuned models

    Meta's own 0-shot results on bf16 weights. Llama 3.x models have no image support and a 128K context, so those cells are not reported.

Only each model's own maker's number is shown; scores a maker reports for rival models are left out.