Humanity's Last Exam (with tools)

How each maker reported it

• Claude Opus 5.5 Table 8.1.A, p.174

  Claude results use adaptive thinking at max effort, averaged over five trials (Terminal-Bench 4.0 at xhigh). Competitor figures come from the developers' own published results.

• Claude Fable 5.1 Table 8.1.A, p.167

  Fable 5.1 and Mythos 5.1 share weights, so column 0 covers both (likewise Fable 5 / Mythos 5). Claude results use adaptive thinking at max effort, averaged over five trials. Terminal-Bench 4.0 values are from §8.6, p.171 (Mythos 5.1: 60.9%; GPT-5.6 Sol from the public leaderboard). Competitor figures come from the developers' own published results or leaderboards.

• Claude Opus 5 Table 8.1.A, p.148–149

  Claude Opus 5 results use adaptive thinking at max effort, averaged over five trials (ARC-AGI-3 at high effort). GPT-5.6 Sol's FrontierBench figure uses Codex and its ARC-AGI-1 figure is at xhigh. The Fable 5 HealthBench Professional figure carries a footnote reading "Mythos 5". Competitor figures come from the developers' own published results or leaderboards.

• Claude Sonnet 5.5 Table 8.1.A, p.109

  Claude results use adaptive thinking at max effort with default sampling, averaged over five trials. Competitor figures come from the developers' own published system cards or leaderboards.

• DeepSeek-V4.1-Flash Table 3, p.33

  All models at max reasoning effort. Code-agent benchmarks use DeepSeek Harness Minimal mode with 1M context; DeepSWE uses mini-SWE; visual agent tasks use the Claude Code harness with 512K context. HLE for some models is the text-only subset. The Hugging Face card lists NL2Repo-Bench as 64.0; the report and change log say 65.4.

• Muse Spark 1.1 model page; report §5

  Highest reasoning effort for every model (high for Gemini 3.1 Pro, max for Opus 4.8). "Muse Spark (1.0)" is the earlier model. Third-party scores are Meta's best-effort runs.

• DeepSeek-V4-Pro model card (0813)

  Code-agent tasks use DeepSeek Harness minimal mode at max effort, temperature 1.0, top_p 0.95. DSBench rows are DeepSeek internal test sets. Competitor figures as published by DeepSeek.

• Claude Opus 4.8 Table 8.1.A, p.194–195

  Opus 4.8 uses adaptive thinking at max effort with default sampling, averaged over 5 trials. Competitor figures come from the developers' own published cards or leaderboards.

• Claude Haiku 5.5 Table 8.1.A, p.111

  Haiku 5.5 results use adaptive thinking at max effort with default sampling, averaged over five trials. FrontierCode is shown at max effort (Sonnet 5.5 scores 52.1 at xhigh). OSWorld is the offline subset; GPT-6 Luna was run by Anthropic on the same tasks. Other competitor figures come from the developers' own published results.

• Claude Sonnet 5 Table 8.1.A, p.115–116; SWE-bench variants §8.2, p.116

  Claude Sonnet 5 results use adaptive thinking at max effort, averaged over five trials. BrowseComp is the single-agent score (86.6% multi-agent) with a 10M-token limit and compaction. GPT-5.5 Terminal-Bench 2.1 figure uses Codex CLI. Competitor figures come from the developers' own published results.

• GPT-6 Astra Launch post (openai.com/index/gpt-6-astra); system card §6, §10.1; GPT-6.1 Sol addendum §5, §9

  Coding and reasoning rows come from the GPT-6 Astra launch post (3 Sep 2026); OpenAI's own comparison table, settings as published there. The system card reports safety and Preparedness evaluations, not general capability benchmarks. Cyber and bio scores are at maximum reasoning effort; bio scores are refusal-adjusted. HealthBench scores are length-adjusted. GPT-6.1 Sol and some GPT-5.6 Sol values come from the GPT-6.1 Sol addendum (29 Sep 2026). Unverified: openai.com blocks automated access, so the coding and reasoning figures were read from copies of the launch post, and they don't appear in the system card or developer docs.

• Qwen3.8-Max model card (Benchmark Results)

  Qwen's own table. Terminal Bench 2.1 run with Claude Code (avg@10); competitor scores are the best published across harnesses. SWE-bench Pro uses a corrected task set. Rows marked in-house are Qwen benchmarks. Fable 5 results may involve fallbacks. Agents' Last Exam is split into pass rate and score.

• Claude Opus 4.7 Table 8.1.A, p.191–192

  Opus 4.7 uses adaptive thinking at max effort with default sampling, averaged over 5 trials (Terminal-Bench 2.0 with thinking disabled; Finance Agent at high effort). Competitor figures come from the developers' own published cards or leaderboards.

• Claude Sonnet 4.6 Table 2.1.A, p.15

  Claude Sonnet 4.6 results are averaged over 10 trials (SWE-bench over 25) with adaptive thinking at max effort and default sampling; Terminal-Bench 2.0 at default thinking. ARC-AGI-2 is 60.4% at high effort. Competitor columns as reported in the card.

• gpt-oss-120b Figures 1, 2, 4; Table 2

  OpenAI's own pass@1 results at reasoning level high. AIME and HLE with tools; Codeforces with a terminal tool. o4-mini and o3 values read from OpenAI's charts.

• gpt-oss-20b Figures 1, 2, 4; Table 2

  OpenAI's own pass@1 results at reasoning level high. AIME and HLE with tools; Codeforces with a terminal tool. o3-mini was run without tools on AIME, HLE and Codeforces, so those cells are left empty. Baseline values read from OpenAI's charts.