Benchmarks
459 benchmarks reported across the cards. Each score is the maker's own number for its own model, with the card's page reference. Makers use different settings and harnesses, so read cross-maker gaps as rough.
Reported by more than one model · 181
- GPQA Diamond21 models · top 94.3% · Gemini 3.1 Pro
- Humanity's Last Exam (no tools)18 models · top 64.4% · Claude Opus 5.5
- SWE-bench Pro17 models · top 89.9% · Claude Opus 5.5
- Humanity's Last Exam (with tools)16 models · top 67.7% · Claude Opus 5.5
- SWE-bench Verified15 models · top 95.0% · Claude Fable 5
- HealthBench Professional14 models · top 69.2% · Claude Sonnet 5.5
- Terminal-Bench 2.112 models · top 90.6% · DeepSeek-V4.1-Flash
- DeepSWE v1.111 models · top 75.4% · Muse Spark 1.3
- GDPval-AA v211 models · top 1861 Elo · Claude Opus 5
- HealthBench11 models · top 62.6 score · GPT-5.4
- HealthBench Hard11 models · top 40.1 score · GPT-5.4
- SWE-bench Multilingual11 models · top 93.9% · Claude Opus 5.5
- AutomationBench10 models · top 59.9% · Mistral Large 4
- OSWorld-Verified10 models · top 85.0% · Claude Fable 5
- MCP-Atlas9 models · top 90.3% · Muse Spark 1.2
- SWE-bench Multimodal8 models · top 61.4% · Claude Opus 5.5
- TroubleshootingBench8 models · top 63.5% · GPT-6 Astra
- τ²-bench Retail7 models · top 91.9% · Claude Opus 4.6
- ARC-AGI-26 models · top 90.4% · Claude Opus 5
- BrowseComp6 models · top 90.8% · Claude Opus 5
- HealthBench Consensus6 models · top 96.6 score · GPT-5.4
- JobBench6 models · top 64.9 Score · Muse Spark 1.3
- LiveCodeBench v66 models · top 91.9% · Qwen3.8-Flash
- MGSM6 models · top 92.3% · Llama 4 Maverick
- MMMLU6 models · top 91.5% · Claude Opus 4.7
- MMMU6 models · top 85.4% · Seed 2.0 Pro
- NL2Repo-Bench6 models · top 65.4% · DeepSeek-V4.1-Flash
- SimpleQA6 models · top 54.0% · Gemini 2.5 Pro
- Tacit Knowledge and Troubleshooting6 models · top 88.5% · GPT-6.1 Sol
- Terminal-Bench 4.06 models · top 66.4% · Claude Opus 5.5
- AIME 20255 models · top 98.3% · Seed 2.0 Pro
- DeepSWE 1.15 models · top 58.7% · Qwen3.8-Flash
- ExploitGym5 models · top 42.4% · GPT-6 Astra
- IFBench5 models · top 82.8% · Qwen3.8-Max
- MMLU5 models · top 90.2% · GPT-4.1
- MMLU Pro5 models · top 87.7% · Seed 2.0 Lite
- MMMU Pro5 models · top 81.2% · Gemini 3 Flash
- MRCR v2 8-needle 128K (average)5 models · top 84.9% · Gemini 3.1 Pro
- Multimodal Troubleshooting Virology5 models · top 63.1% · GPT-6 Astra
- OSWorld 2.0 (partial)5 models · top 81.8% · Claude Opus 5.5
- ProtocolQA Open-Ended5 models · top 44.4% · GPT-6 Sol
- SEC-Bench Pro5 models · top 85.4% · GPT-6 Astra
- AA-Briefcase4 models · top 1720 Elo · Claude Opus 5
- Agents' Last Exam (score)4 models · top 52.4 Score · Qwen3.8-Max
- CharXiv Reasoning (no tools)4 models · top 86.2% · Gemini 3.8 Flash
- CoWorkBench (in-house)4 models · top 74.8% · Qwen3.8-Max
- ExploitBench4 models · top 100.0% · GPT-6 Astra
- ExploitBench Internal Port (Jun–Aug 2026)4 models · top 31.5% · GPT-6 Astra
- FACTS Grounding4 models · top 87.8% · Gemini 2.5 Pro
- FrontierCode v1.1 (Main)4 models · top 54.4% · Claude Opus 5.5
- GDPval-AA4 models · top 1932 Elo · Claude Fable 5
- HLE4 models · top 43.6% · Qwen3.8-Max
- Stylization4 models · top 1098 Elo · Nano Banana Pro
- Terminal-Bench 2.04 models · top 69.4% · Claude Opus 4.7
- AA-Briefcase v1.13 models · top 1822 Elo · Claude Opus 5.5
- Agents' Last Exam (Pass@1)3 models · top 24.3% · Qwen3.8-Flash
- AIME 2024 (Reasoning 2512 checkpoint)3 models · top 89.8% · Ministral 3 14B
- AIME 2025 (Reasoning 2512 checkpoint)3 models · top 85.0% · Ministral 3 14B
- AndroidWorld3 models · top 84.5% · Qwen3.8-Flash
- ARC-AGI-13 models · top 97.5% · Claude Fable 5.1
- ARC-AGI-2 (Verified)3 models · top 68.8% · Claude Opus 4.6
- Arena Hard3 models · top 55.1% · Ministral 3 14B
- Character editing3 models · top 1176 Elo · Nano Banana Pro
- CharXiv Reasoning3 models · top 88.4% · Muse Spark 1.1
- CharXiv Reasoning (with tools)3 models · top 91.0% · Claude Opus 4.7
- CharXiv RQ (no code interpreter)3 models · top 85.8% · Qwen3.7-Plus
- CharXiv RQ (with code interpreter)3 models · top 90.6% · Qwen3.8-Flash
- ClawEval-MM (average)3 models · top 60.4% · Qwen3.8-Flash
- ClawEval-MM (Pass@3)3 models · top 64.4% · Qwen3.8-Flash
- Codeforces3 models · top 3471 Elo · DeepSeek-V4.1-Flash
- CVE-Bench3 models · top 93.1% · GPT-5.5
- CyberGym3 models · top 88.1% · DeepSeek-V4.1-Flash
- CyberGym (mean reproduced)3 models · top 80.4% · Grok 4.5
- Destructive actions: avoidance + correctness3 models · top 0.44 score · GPT-5.6 Sol
- DROP (F1, 3-shot)3 models · top 81 F1 · GPT-4.1 Mini
- ERQA3 models · top 72.3% · Qwen3.8-Flash
- GDPval-AA v2.13 models · top 1846 Elo · Claude Opus 5.5
- General editing3 models · top 1065 Elo · Nano Banana 2
- Global MMLU (Lite)3 models · top 89.2% · Gemini 2.5 Pro
- GPQA3 models · top 66.3% · GPT-4.1
- GPQA Diamond (no tools)3 models · top 86.9% · Gemini 3.1 Flash-Lite
- GPQA Diamond (Reasoning 2512 checkpoint)3 models · top 71.2% · Ministral 3 14B
- HumanEval3 models · top 94.5% · GPT-4.1
- LiveCodeBench (Reasoning 2512 checkpoint)3 models · top 64.6% · Ministral 3 14B
- MATH3 models · top 82.1% · GPT-4.1
- MATH maj@13 models · top 90.4% · Ministral 3 14B
- MathVision (no code interpreter)3 models · top 90.6% · Qwen3.8-Flash
- MathVision (with code interpreter)3 models · top 95.7% · Qwen3.8-Flash
- MathVista3 models · top 89.8% · Seed 2.0 Pro
- MentalHealthBench (overall)3 models · top 58.7% · GPT-6 Astra
- MLE-Bench3 models · top 63.9% · Gemini 3.6 Flash
- MM MTBench3 models · top 8.49 Score · Ministral 3 14B
- MMMU-Pro (no tools)3 models · top 76.8% · Gemini 3.1 Flash-Lite
- Monorepo-Bench3 models · top 60.0% · GPT-5.5
- MRCR v2 8-needle, 1M pointwise3 models · top 16.4% · Gemini 2.5 Pro
- Object/environment editing3 models · top 1102 Elo · Nano Banana Pro
- OfficeQA Pro3 models · top 80.6% · Claude Opus 4.7
- Prompt injection: connectors3 models · top 1 score · GPT-5.6 Terra
- Prompt injection: search & function calling3 models · top 0.946 score · GPT-5.6 Terra
- RealWorldQA3 models · top 88.5% · Qwen3.8-Flash
- RecreationBench (in-house)3 models · top 49.9% · Qwen3.8-Flash
- SimpleQA Verified3 models · top 43.3% · Gemini 3.1 Flash-Lite
- SuperGPQA3 models · top 68.7% · Seed 2.0 Pro
- SWE-Bench Pro (Public)3 models · top 58.7% · Gemini 3.6 Flash
- Terminal-Bench 2.0 (Terminus-2)3 models · top 68.5% · Gemini 3.1 Pro
- Terminal-Bench-Science 0.13 models · top 64.6% · GPT-6 Astra
- Toolathlon Verified (Pass@1)3 models · top 73.5% · Qwen3.8-Flash
- Vibe-Eval (Reka)3 models · top 67.2% · Gemini 2.5 Pro
- Vision2Web3 models · top 64.0% · Qwen3.8-Flash
- WildBench3 models · top 68.5 Score · Ministral 3 14B
- τ²-bench Telecom3 models · top 99.3% · Claude Opus 4.6
- 3DCodeBench2 models · top 54.0% · Grok 4.6
- AA-LCR2 models · top 80.0% · Muse Glimmer 30B
- Agents' Last Exam2 models · top 31.8% · DeepSeek-V4.1-Flash
- AIME 2025 (with tools)2 models · top 98.7% · gpt-oss-20b
- APEX-Agents2 models · top 57.5% · Grok 4.6
- Audio prompt following2 models · top 3.7 /5 · Dreamina Seedance 2.0
- Audio-visual sync2 models · top 3.75 /5 · Dreamina Seedance 2.0
- BFCL-v42 models · top 73.4% · Seed 2.0 Pro
- CADGenBench (generation)2 models · top 44.4% · Grok 4.7
- Capture the Flag (Professional)2 models · top 96.3% · GPT-5.5
- ChartQA2 models · top 90.0% · Llama 4 Maverick
- CloningScenarios2 models · top 67.0% · Grok 4.20
- Codeforces (with tools)2 models · top 2622 Elo · gpt-oss-120b
- CyBench (unguided success)2 models · top 53.0% · Grok 4.20
- Cyber Range (combined pass rate)2 models · top 93.3% · GPT-5.5
- DocVQA (test)2 models · top 94.4 ANLS · Llama 4 Scout
- Doodle editing2 models · top 1106 Elo · Nano Banana Pro
- EEBench2 models · top 66.0% · Grok 4.7
- FactScore2 models · top 71.2% · Seed 2.0 Pro
- FigQA2 models · top 66.0% · Grok 4.20
- Finance Agent v22 models · top 57.2% · Muse Spark 1.1
- GDM-MRCR v2 8-needle, 128K (average)2 models · top 91.8% · Gemini 3.6 Flash
- GDM-MRCR v2 8-needle, 1M (pointwise)2 models · top 54.0% · Gemini 3.6 Flash
- HLE (no tools, text only)2 models · top 32.4% · Seed 2.0 Pro
- HumanEval (pass@1)2 models · top 88.4% · Llama 3.3 70B
- IFEval2 models · top 92.1% · Llama 3.3 70B
- IMOAnswerBench (no tool)2 models · top 89.3% · Seed 2.0 Pro
- Internal Research Debugging2 models · top 78.0% · GPT-6 Astra
- LiveCodeBench (1/1/2025–5/1/2025)2 models · top 72.0% · Gemini 3.1 Flash-Lite
- LiveCodeBench (10/2024-02/2025)2 models · top 43.4% · Llama 4 Maverick
- LongBench v2 (128K)2 models · top 63.8% · Seed 2.0 Pro
- LVBench (long video)2 models · top 76.6% · Qwen3.8-Flash
- MakeMeSay win rate2 models · top 8.0% · Grok 4.20
- MATH (CoT)2 models · top 77.0% · Llama 3.3 70B
- MMLU (CoT, 0-shot)2 models · top 86.0% · Llama 3.3 70B
- MMLU Pro (CoT, 5-shot)2 models · top 68.9% · Llama 3.3 70B
- MMMLU (14-language average)2 models · top 81.3% · gpt-oss-120b
- MMMU Pro (avg of Standard and Vision)2 models · top 59.6% · Llama 4 Maverick
- MMMU-Pro (with tools)2 models · top 77.3% · Claude Opus 4.6
- Motion quality2 models · top 3.75 /5 · Dreamina Seedance 2.0
- MRCR v2 (8-needle)2 models · top 54.0% · Seed 2.0 Pro
- MTOB full book, eng→kgv2 models · top 50.8 chrF · Llama 4 Maverick
- MTOB full book, kgv→eng2 models · top 46.7 chrF · Llama 4 Maverick
- MTOB half book, eng→kgv2 models · top 54 chrF · Llama 4 Maverick
- MTOB half book, kgv→eng2 models · top 46.4 chrF · Llama 4 Maverick
- Multi-input (1–3)2 models · top 1109 Elo · Nano Banana Pro
- Multi-select multimodal troubleshooting virology2 models · top 50.4% · GPT-5.4
- MultiChallenge2 models · top 68.3% · Seed 2.0 Pro
- OpenAI-Proof Q&A2 models · top 4.2% · GPT-5.4
- OSWorld2 models · top 78.0% · Claude Opus 4.7
- OSWorld 2.0 (binary)2 models · top 19.4% · Qwen3.8-Flash
- OSWorld 2.0 (strict)2 models · top 48.7% · Claude Opus 5.5
- ProtocolQA2 models · top 79.0% · Grok 4.20
- QwenSWEBench (in-house)2 models · top 80.7% · Qwen3.8-Max
- ScreenSpot-Pro (no tools)2 models · top 82.3% · Claude Opus 4.8
- ScreenSpot-Pro (with tools)2 models · top 87.9% · Claude Opus 4.8
- SpreadsheetBench Verified2 models · top 82.3% · Seed 2.0 Lite
- SWE-Atlas-QnA2 models · top 84.0% · Grok 4.5
- SWE-bench Verified (single attempt)2 models · top 59.6% · Gemini 2.5 Pro
- SWE-Marathon v1.12 models · top 46.0% · Grok 4.7
- Terminal-bench 2.1 (Terminus-2)2 models · top 78.0% · Gemini 3.6 Flash
- Terminal-Bench 3.02 models · top 30.0% · DeepSeek-V4.1-Flash
- Text editing2 models · top 1202 Elo · Nano Banana Pro
- Toolathlon-Verified2 models · top 75.6% · Muse Spark 1.1
- VCT2 models · top 54.0% · Grok 4.20
- Video prompt following2 models · top 3.46 /5 · Dreamina Seedance 2.0
- Visual design2 models · top 1104 Elo · Nano Banana Pro
- WMDP Bio2 models · top 91.0% · Grok 4.20
- WMDP Chem2 models · top 90.0% · Grok 4.20
- WMDP Cyber2 models · top 91.0% · Grok 4.20
Reported by one model · 278
- $OneMillion-Bench (expert score)1 model · top 52.5 Score · Qwen3.8-Max
- AAV Capsid Packaging Prediction (Spearman)1 model · top 0.529 ρ · GPT-5.6 Sol
- Aesthetics1 model · top 3.67 /5 · Dreamina Seedance 2.0
- Agent Startup Bench1 model · top 54.0% · Dola Seed 2.1 Turbo
- Agentic IF Index (Meta internal)1 model · top 57.8 Score · Muse Spark 1.3
- Agents' Last Exam (pass)1 model · top 27.0% · Qwen3.8-Max
- Aider Polyglot1 model · top 82.2% · Gemini 2.5 Pro
- Aider Polyglot (whole)1 model · top 26.7% · Gemini 2.5 Flash-Lite
- AIME 20261 model · top 94.7% · Muse Glimmer 30B
- AIME 2026 (no tools)1 model · top 89.2% · Gemma 4 31B
- AndroidBench1 model · top 75.1% · Qwen3.8-Max
- APEX-SWE1 model · top 56.4% · Grok 4.6
- API-Bank1 model · top 82.6% · Llama 3.1 8B
- ARC-AGI-2 (ARC Prize Verified)1 model · top 77.1% · Gemini 3.1 Pro
- ARC-AGI-31 model · top 30.2% · Claude Opus 5
- ARC-C (0-shot)1 model · top 83.4% · Llama 3.1 8B
- ARC-Easy, English1 model · top 93.9% · GPT-4o Mini
- ARC-Easy, Hausa1 model · top 58.5% · GPT-4o Mini
- ARC-Easy, Yoruba1 model · top 43.8% · GPT-4o Mini
- Audio quality1 model · top 3.63 /5 · Dreamina Seedance 2.0
- Audio quality & expressiveness1 model · top 3.61 /5 · Dreamina Seedance 2.0
- Automation-Bench (Pass@1)1 model · top 27.3% · Qwen3.8-Max
- AutomationBench (Public)1 model · top 31.8% · DeepSeek-V4-Pro
- Average WER, non-streaming (Artificial Analysis)1 model · top 2.6% · Gemini 3.5 Transcribe
- Average WER, streaming (Artificial Analysis)1 model · top 4.0% · Gemini 3.5 Transcribe
- BabyVision1 model · top 76.3% · Muse Spark 1.1
- BabyVision (no code interpreter)1 model · top 65.7% · Qwen3.8-27B
- BabyVision (with code interpreter)1 model · top 85.6% · Qwen3.8-27B
- BabyVision (with tools)1 model · top 89.6% · DeepSeek-V4.1-Flash
- Batch (pre-recorded)1 model · top 5.3% · Deepgram Nova-3
- Beam128K1 model · top 65.1% · Muse Glimmer 30B
- BeyondAIME1 model · top 88.0% · Dola Seed 2.1 Turbo
- BFCL1 model · top 76.1% · Llama 3.1 8B
- BFCL v21 model · top 77.3% · Llama 3.3 70B
- BIG-Bench Extra Hard (0-shot)1 model · top 19.3% · Gemma 3 27B
- BIG-Bench Hard (0-shot)1 model · top 87.6% · Gemma 3 27B
- BigBench Extra Hard1 model · top 74.4% · Gemma 4 31B
- Biochemistry knowledge1 model · top 32.3% · GPT-5.5
- Bird-SQL (dev)1 model · top 54.4% · Gemma 3 27B
- Blind head-to-head preference vs 4 competitors1 model · top 75.0% · Eleven v4
- Blind human coding eval (Surge AI)1 model · top 3.74 /5 · Mistral Large 4
- Blueprint-Bench 21 model · top 38.6% · Claude Fable 5
- BrowseComp (multi-agent)1 model · top 88.5% · Claude Opus 4.8
- BrowseComp (single-agent)1 model · top 84.3% · Claude Opus 4.8
- BrowseComp (with search)1 model · top 84.9% · Dola Seed 2.1 Turbo
- CAD-Bench1 model · top 88.1% · Grok 4.5
- Capture the Flag1 model · top 81.3% · GPT-5.4 Mini
- Capture the Flag (internal, 63 challenges)1 model · top 96.7% · GPT-5.6 Sol
- Chart editing1 model · top 1209 Elo · Nano Banana Pro
- Chartography (with tools)1 model · top 78.9% · DeepSeek-V4.1-Flash
- ChartQAPro (no tools)1 model · top 69.4% · Claude Opus 4.8
- ChartQAPro (with tools)1 model · top 72.3% · Claude Opus 4.8
- Codeforces (no tool)1 model · top 3020 Elo · Seed 2.0 Pro
- Coding Agent Index1 model · top 49.8% · Mistral Large 4
- Common Voice1 model · top 0.139 WER · GPT-4o mini TTS
- Creative1 model · top 1023 Elo · Nano Banana 2
- Creative biology (mean)1 model · top 0.48 score · Claude Sonnet 4.5
- CursorBench 3.21 model · top 69.9% · Grok 4.6
- CursorBench 4.01 model · top 46.3% · Grok 4.7
- Cybench1 model · top 93.0% · Mistral Large 4
- Cybench (32-challenge subset), solved1 model · top 15 of 32 · Claude Haiku 4.5
- DeepSearch QA1 model · top 74.6% · Muse Glimmer 30B
- DeepSearchQA1 model · top 90.3 F1 · Muse Spark 1.3
- DeepSearchQA (internal implementation)1 model · top 81.6% · Grok 4.6
- DeepSWE1 model · top 62.7% · DeepSeek-V4-Pro
- DeepSWE v1.01 model · top 62.0% · Grok 4.5
- Dense 200 (visual grounding)1 model · top 42.0% · Mistral Large 4
- Designer preference (internal arena)1 model · top 1062 Elo · Ideogram 4.0
- DNA design for TF binding1 model · top 13.8% · GPT-5.5
- DNA sequence design for TF binding (pass@1)1 model · top 13.7% · GPT-5.6 Sol
- DSBench-FullStack (internal)1 model · top 71.1% · DeepSeek-V4-Pro
- DSBench-Hard (internal)1 model · top 67.2% · DeepSeek-V4-Pro
- Editing consistency (1–5)1 model · top 3.54 /5 · Dreamina Seedance 2.0
- Elo1 model · top 1247 Elo · Runway Gen-4.5
- EndOfTurn latency, p901 model · top 1 s · Deepgram Flux
- EndOfTurn latency, p951 model · top 1.5 s · Deepgram Flux
- FACTS Benchmark Suite1 model · top 40.6% · Gemini 3.1 Flash-Lite
- Factuality – Edu1 model · top 1169 Elo · Nano Banana Pro
- Fewer false interruptions vs pipeline approaches1 model · top 30.0% · Deepgram Flux
- Finance Agent1 model · top 64.4% · Claude Opus 4.7
- FLEURS1 model · top 0.057 WER · GPT-4o mini TTS
- FLEURS WER, non-streaming (top languages)1 model · top 5.0% · Gemini 3.5 Transcribe
- FLEURS WER, streaming (top languages)1 model · top 5.5% · Gemini 3.5 Transcribe
- FrontierBench v0.11 model · top 43.3% · Claude Opus 5
- FrontierCode (Diamond)1 model · top 29.3% · Claude Fable 5
- FrontierCode 1.1 (Main)1 model · top 53.4% · Claude Opus 5
- FrontierCode v11 model · top 38.8% · Claude Sonnet 5
- FrontierCode v1.1 (extended)1 model · top 61.3% · Grok 4.6
- FrontierScience-Olympiad1 model · top 76.0% · Dola Seed 2.1 Turbo
- FrontierSWE1 model · top 73.5% · Qwen3.8-Max
- FrontierSWE (dominance)1 model · top 78.0% · Grok 4.5
- FrontierSWE V2 (mean@5)1 model · top 29.0% · Grok 4.7
- Gaia21 model · top 43.3% · Muse Glimmer 30B
- GDP.pdf (strict pass)1 model · top 29.8% · Claude Fable 5
- GDPval1 model · top 82.7% · Dola Seed 2.1 Turbo
- General image editing1 model · top 1127 Elo · Nano Banana Pro
- General T2I1 model · top 1059 Elo · Nano Banana 2 Lite
- General text-to-image1 model · top 1094 Elo · Nano Banana Pro
- GPQA (0-shot)1 model · top 30.4% · Llama 3.1 8B
- GPQA Diamond (0-shot)1 model · top 42.4% · Gemma 3 27B
- GPQA Diamond (AA)1 model · top 83.5% · Muse Glimmer 30B
- GPQA Diamond (CoT)1 model · top 50.5% · Llama 3.3 70B
- GraphWalks BFS 256K1 model · top 85.9% · Claude Opus 4.8
- GraphWalks Parents 256K1 model · top 99.3% · Claude Opus 4.8
- GSM-8K (CoT)1 model · top 84.5% · Llama 3.1 8B
- GSM8K (0-shot)1 model · top 95.9% · Gemma 3 27B
- Hard-negative protein binding1 model · top 1.5% · GPT-5.5
- Hard-negative protein binding prediction (pass@4)1 model · top 7.6% · GPT-5.6 Sol
- Harvey's Legal Agent Benchmark (all pass)1 model · top 10.0% · Gemini 3.8 Flash
- HealthBench Professional (length-adjusted)1 model · top 64.8% · Claude Haiku 5.5
- HiddenMath (0-shot)1 model · top 60.3% · Gemma 3 27B
- HLE (with search)1 model · top 26.5% · Gemma 4 31B
- HLE Text (AA)1 model · top 22.0% · Muse Glimmer 30B
- HLE text-only (with search)1 model · top 54.6% · Dola Seed 2.1 Turbo
- HLE-text (with tools)1 model · top 54.2% · Seed 2.0 Pro
- HLE-Verified1 model · top 54.9% · Gemini 3.8 Flash
- HLE-Verified (no tool)1 model · top 42.4% · Dola Seed 2.1 Turbo
- HumanEval (0-shot)1 model · top 87.8% · Gemma 3 27B
- Humanity's Last Exam (search + code)1 model · top 51.4% · Gemini 3.1 Pro
- IFEval (0-shot)1 model · top 90.4% · Gemma 3 27B
- Image preservation1 model · top 3.31 /5 · Dreamina Seedance 2.0
- Image-to-video Elo (720p, 8 Apr 2026)1 model · top 1449 Elo · Dreamina Seedance 2.0
- Infographic design1 model · top 1048 Elo · Nano Banana 2.1
- Infographic factuality1 model · top 0.521 score · Nano Banana 2.1
- Infographics1 model · top 1268 Elo · Nano Banana Pro
- Infographics (factuality)1 model · top 1114 Elo · Nano Banana 2
- Internal AI research suite 2 (threshold 0.6)1 model · top 0.514 score · Claude Sonnet 4.5
- Internal Research Debugging (median)1 model · top 50.5% · GPT-5.5
- Irregular CyScenarioBench (avg success)1 model · top 28.0% · GPT-5.6 Sol
- Kernel optimization, hard variant (best run)1 model · top 108.64 × speedup · Claude Sonnet 4.5
- KernelBenchInternal v1.11 model · top 37.2% · Grok 4.6
- LAB-Bench Cloning Scenarios1 model · top 0.45 score · Claude Haiku 4.5
- LAB-Bench FigQA1 model · top 0.49 score · Claude Haiku 4.5
- LAB-Bench ProtocolQA1 model · top 0.69 score · Claude Haiku 4.5
- LAB-Bench SeqQA1 model · top 0.66 score · Claude Haiku 4.5
- LABBench21 model · top 86.2% · Gemini 3.8 Flash
- LatchBio Capabilities v1.01 model · top 44.5% · Grok 4.7
- Layout control (7Bench mIoU)1 model · top 0.69 score · Ideogram 4.0
- Legal Agent Benchmark1 model · top 15.8% · Grok 4.6
- Legal Agent Benchmark (full public set)1 model · top 8.9% · Claude Sonnet 5
- Legal Agent Benchmark (Harvey final score)1 model · top 19.6% · Grok 4.7
- Legal Agent Benchmark (Harvey held-out set)1 model · top 5.8% · Claude Sonnet 5
- Legal Agent Benchmark (Harvey held-out)1 model · top 13.3% · Claude Fable 5
- LEXam-hard1 model · top 31.89 score · Mistral Medium 3.5
- LiveCodeBench (0-shot)1 model · top 29.7% · Gemma 3 27B
- LiveCodeBench Pro1 model · top 2887 Elo · Gemini 3.1 Pro
- LiveCodeBench v51 model · top 63.9% · Gemini 2.5 Flash
- LiveCodeBench v5 (1/1/2025–5/1/2025)1 model · top 33.7% · Gemini 2.5 Flash-Lite
- LLM training optimization (average best)1 model · top 5.5 × speedup · Claude Sonnet 4.5
- Long-Form Virology Task 11 model · top 0.76 score · Claude Haiku 4.5
- Long-form virology task 1 (max of 5)1 model · top 0.89 score · Claude Sonnet 4.5
- Long-Form Virology Task 21 model · top 0.17 score · Claude Haiku 4.5
- Long-form virology task 2 (pass@5)1 model · top 0.88 score · Claude Sonnet 4.5
- LongBench v21 model · top 66.3% · Qwen3.8-Max
- LVBench1 model · top 87.8% · Gemini 3.8 Flash
- Mask/ink-based editing1 model · top 1049 Elo · Nano Banana 2.1
- MATH (0-shot)1 model · top 89.0% · Gemma 3 27B
- MATH-Vision1 model · top 85.6% · Gemma 4 31B
- MathArena Apex1 model · top 65.6% · DeepSeek-V4.1-Flash
- MBPP (3-shot)1 model · top 74.4% · Gemma 3 27B
- MBPP EvalPlus (base)1 model · top 87.6% · Llama 3.3 70B
- MBPP++ base (pass@1)1 model · top 72.8% · Llama 3.1 8B
- MCP Atlas (Public)1 model · top 75.5% · Muse Glimmer 30B
- MCP-Mark1 model · top 54.7% · Seed 2.0 Pro
- MedXPertQA MM1 model · top 61.3% · Gemma 4 31B
- Meta Internal Coding Bench1 model · top 70.6% · Muse Spark 1.2
- MGSM (CoT)1 model · top 68.9% · Llama 3.1 8B
- MLE-Bench-301 model · top 36.7% · GPT-5.5
- MLS-Bench-Lite1 model · top 41.0% · Qwen3.8-Max
- MMLU (5-shot)1 model · top 69.4% · Llama 3.1 8B
- MMLU-Pro (0-shot)1 model · top 67.5% · Gemma 3 27B
- MMMU (validation)1 model · top 80.7% · Claude Opus 4.5
- MRCR v2 256K (8-needle)1 model · top 92.9% · Qwen3.8-Max
- MRCR v2 8-needle (256K–512K)1 model · top 98.5 Score · Muse Spark 1.3
- MRCR v2 8-needle (512K–1M)1 model · top 98.1 Score · Muse Spark 1.3
- MRCR v2, 128K average1 model · top 74.0% · Gemini 2.5 Flash
- MRCR v2, 1M pointwise1 model · top 32.0% · Gemini 2.5 Flash
- MSQA1 model · top 42.0% · Dola Seed 2.1 Turbo
- Multi input (up to 5)1 model · top 980 Elo · Nano Banana 2 Lite
- Multi-character (up to 5) editing1 model · top 1020 Elo · Nano Banana 2 Lite
- Multi-character consistency1 model · top 1106 Elo · Nano Banana 2.1
- Multi-character editing1 model · top 1213 Elo · Nano Banana Pro
- Multi-image editing1 model · top 2 rank · Muse Image 1.0
- Multi-product (up to 14) editing1 model · top 1029 Elo · Nano Banana 2 Lite
- Multi-reference editing1 model · top 1066 Elo · Nano Banana 2.1
- Multi-select virology troubleshooting1 model · top 53.7% · GPT-5.5
- Multi-turn1 model · top 1186 Elo · Nano Banana Pro
- Multilingual LibriSpeech1 model · top 0.043 WER · GPT-4o mini TTS
- Multimodal task following (1–3)1 model · top 2.5 /3 · Dreamina Seedance 2.0
- Multimodal virology (VCT, 10-shot)1 model · top 39.7% · Claude Sonnet 4.5
- Natural2Code (0-shot)1 model · top 84.5% · Gemma 3 27B
- NL2Repo1 model · top 61.5 Score · DeepSeek-V4-Pro
- Novel compiler, advanced tests (mean pass rate)1 model · top 29.7% · Claude Sonnet 4.5
- Novel compiler, basic tests (mean pass rate)1 model · top 81.7% · Claude Sonnet 4.5
- OCRBench v21 model · top 62.5% · Seed 2.0 Pro
- OfficeQA1 model · top 86.3% · Claude Opus 4.7
- OfficeQA Pro (Databricks)1 model · top 57.9% · Claude Fable 5
- OmniDocBench 1.51 model · top 91.1% · Qwen3.8-27B
- OmniDocBench 1.5 (edit distance, lower is better)1 model · top 0.131 score · Gemma 4 31B
- OmniDocBench v1.51 model · top 75.8% · Muse Glimmer 30B
- OpenAI-Proof Q&A v01 model · top 7.5% · GPT-5.4 Mini
- OSWorld 2.01 model · top 70.6% · Claude Opus 5
- OSWorld 2.0 (binary completion)1 model · top 32.0% · Muse Spark 1.3
- OSWorld 2.0 (partial score)1 model · top 66.9 Score · Muse Spark 1.3
- OSWorld 2.11 model · top 80.1% · Claude Sonnet 5.5
- OSWorld 2.1 (offline subset)1 model · top 72.4% · Claude Haiku 5.5
- Output speed1 model · top 363 tokens/s · Gemini 3.1 Flash-Lite
- Overall preference1 model · top 1050 Elo · Nano Banana 2.1
- Overall preference (GenAI-Bench)1 model · top 1079 Elo · Nano Banana 2
- PaperBench1 model · top 93.0% · Qwen3.8-Max
- PLawBench1 model · top 73.2% · Qwen3.8-Max
- PRBench-Finance1 model · top 58.3% · Qwen3.8-Max
- PRBench-Legal1 model · top 57.6% · Qwen3.8-Max
- Product consistency1 model · top 1024 Elo · Nano Banana 2.1
- ProgramBench (Almost@1)1 model · top 20.3% · DeepSeek-V4.1-Flash
- ProgramBench (tests passed)1 model · top 57.2% · Grok 4.5
- Prompt alignment (PRISM-Bench)1 model · top 0.89 score · Ideogram 4.0
- Quadruped RL, easy variant (highest, threshold 1)1 model · top 1.302 score · Claude Sonnet 4.5
- QwenQoderBench (in-house)1 model · top 58.4% · Qwen3.8-Max
- QwenReactBench (in-house)1 model · top 1724 Elo · Qwen3.8-Max
- QwenSVGBench (in-house)1 model · top 1713 Elo · Qwen3.8-Max
- Regression sets / popular use cases1 model · top 1128 Elo · Nano Banana Pro
- Reproduce-and-patch1 model · top 82.0% · Mistral Large 4
- SandboxBench1 model · top 45.5% · GPT-6 Astra
- SandboxBench (targets of 22)1 model · top 1 targets · GPT-6 Sol
- SciCode1 model · top 43.6% · Muse Glimmer 30B
- ScreenSpot Pro1 model · top 75.4% · Muse Glimmer 30B
- SimpleQA (0-shot)1 model · top 10.0% · Gemma 3 27B
- Single-character consistency1 model · top 1028 Elo · Nano Banana 2.1
- Single-image editing1 model · top 2 rank · Muse Image 1.0
- SkillsBench1 model · top 70.2% · Qwen3.8-Max
- SkillsBench (with skills)1 model · top 44.3% · Muse Glimmer 30B
- Social media trends – editing1 model · top 996 Elo · Nano Banana 2 Lite
- Social media trends – T2I1 model · top 1030 Elo · Nano Banana 2 Lite
- Spatial reasoning (SpatialGenEval)1 model · top 0.76 score · Ideogram 4.0
- SRE-Bench (pass@4)1 model · top 99.2% · GPT-6 Astra
- Streaming1 model · top 6.8% · Deepgram Nova-3
- SWE-Atlas1 model · top 30.6% · Dola Seed 2.1 Turbo
- SWE-Atlas Codebase QnA1 model · top 59.4% · Muse Spark 1.3
- SWE-bench Verified (hard subset, >1 h tasks)1 model · top 45.3% · Claude Sonnet 4.5
- SWE-bench Verified (hard subset), pass@11 model · top 36.6% · Claude Haiku 4.5
- SWE-bench Verified (multiple attempts)1 model · top 42.6% · Gemini 2.5 Flash-Lite
- SWE-bench Verified (OpenAI agentic harness)1 model · top 55.0% · GPT-4.1
- SWE-Marathon1 model · top 29.0% · Grok 4.5
- SWE-MM1 model · top 38.6% · Qwen3.8-27B
- SWE-Pro Bench1 model · top 57.0% · Dola Seed 2.1 Turbo
- Tacit Knowledge and Troubleshooting (refusals counted correct)1 model · top 84.1% · GPT-5.6 Terra
- Tau2 (average over 3)1 model · top 76.9% · Gemma 4 31B
- Terminal Bench 2.1 (Terminus)1 model · top 73.0% · Qwen3.8-27B
- Terminal-Bench 41 model · top 28.3% · Mistral Large 4
- TerminalBench 2.1 (terminus2)1 model · top 51.7% · Muse Glimmer 30B
- Text rendering1 model · top 1198 Elo · Nano Banana Pro
- Text rendering (X-Omni English OCR)1 model · top 0.97 score · Ideogram 4.0
- Text-based RL scaffolding (best score)1 model · top 0.85 score · Claude Sonnet 4.5
- Text-reading error rate (internal, 27 categories × 8 languages)1 model · top 4.9% · Eleven v3
- Text-to-image1 model · top 2 rank · Muse Image 1.0
- Text-to-video Elo (720p, 8 Apr 2026)1 model · top 1450 Elo · Dreamina Seedance 2.0
- Toolathlon1 model · top 49.1% · Dola Seed 2.1 Turbo
- TruthfulQA, English1 model · top 66.5% · GPT-4o Mini
- TruthfulQA, Yoruba1 model · top 35.8% · GPT-4o Mini
- Uhura-Eval, Yoruba1 model · top 44.2% · GPT-4o Mini
- UK AISI CTF expert-level tasks1 model · top 95.0% · GPT-5.6 Sol
- User preference, GA vs alpha1 model · top 72.0% · Eleven v3
- Vals Finance Agent v21 model · top 61.4% · Gemini 3.8 Flash
- Video-MMMU1 model · top 84.8% · Gemini 3.1 Flash-Lite
- VideoMME1 model · top 86.9% · Gemini 2.5 Pro
- VideoMMMU1 model · top 83.6% · Gemini 2.5 Pro
- Visual quality (GenAI-Bench)1 model · top 1140 Elo · Nano Banana 2
- VMQA/VCT1 model · top 0.32 score · Claude Haiku 4.5
- WebArena-Verified1 model · top 64.8% · Qwen3.8-27B
- WideSearch (item-F1)1 model · top 81.9% · Qwen3.8-Max
- WildClawBench1 model · top 47.6% · Muse Glimmer 30B
- Workspace Bench (total)1 model · top 54.7% · Dola Seed 2.1 Turbo
- WorkSpaceBench1 model · top 67.7% · Qwen3.8-Max
- ZeroBench-main (with tools, Pass@5)1 model · top 49.0% · DeepSeek-V4.1-Flash
- τ3-Banking1 model · top 23.5% · Muse Glimmer 30B
- τ³-banking (AA)1 model · top 32.6% · Grok 4.5
- τ³-Telecom1 model · top 91.4% · Mistral Medium 3.5