aiaimodel.cards
← All benchmarks

MMMU

College-level questions that require reading images, charts and diagrams.

Your workloadRAG over docs · 12K in · 600 out · 50K requests/mo · 30% cached
  1. 1Seed 2.0 Pro · ByteDancebest value
    85.4%
  2. 2Gemini 2.5 Pro · Google
    82.0%
  3. 3Gemini 2.5 Flash · Googlebest value
    79.7%
  4. 4Llama 4 Maverick · Metabest value
    73.4%
  5. 5Gemini 2.5 Flash-Lite · Googlebest value
    72.9%
  6. 6Llama 4 Scout · Meta
    69.4%

“Best value”: no other model here reports a higher score at a lower cost for your workload. Scores are self-reported by each maker with different settings, so treat close calls as ties.

How each maker reported it

  • Seed 2.0 Pro Tables 3, 8, 11 (pp.9, 13, 16)

    ByteDance's own runs. Competitor columns are as printed in the Seed2.0 card (frontier models available in early 2026). Codeforces is an Elo rating. HLE-text under Agentic is with search tools.

  • Gemini 2.5 Pro p.5–6

    Results as of June 2025, as published in Google's model card. Single-attempt pass@1. Competitor figures are providers' own numbers as listed by Google; Claude 4 Opus is 32K thinking. Aider Polyglot uses diff-fenced for 2.5 Pro and diff for the others. MRCR v2 for Claude 4 Opus is with no thinking.

  • Gemini 2.5 Flash p.5–6

    Results as of May 2025 for Gemini 2.5 Flash GA with thinking, as published in Google's model card. Competitor figures are as listed by Google; Claude 3.7 Sonnet uses 64K extended thinking where available, Grok 3 Beta extended thinking. MRCR v2 here is the earlier (non 8-needle) version. Aider Polyglot and SWE-bench rows are left out because their columns are ambiguous in the PDF text.

  • Llama 4 Maverick Benchmarks: Instruction tuned models

    Meta's own 0-shot results on bf16 weights. Llama 3.x models have no image support and a 128K context, so those cells are not reported.

  • Gemini 2.5 Flash-Lite p.5

    Results for the 06-17 release (the stable model) as of June 2025, as published in Google's model card. Results for the shut-down 09-2025 preview are left out.

  • Llama 4 Scout Benchmarks: Instruction tuned models

    Meta's own 0-shot results on bf16 weights. Llama 3.x models have no image support and a 128K context, so those cells are not reported.

Only each model's own maker's number is shown; scores a maker reports for rival models are left out.