Terminal-Bench 2.1

How each maker reported it

• DeepSeek-V4.1-Flash Table 3, p.33

  All models at max reasoning effort. Code-agent benchmarks use DeepSeek Harness Minimal mode with 1M context; DeepSWE uses mini-SWE; visual agent tasks use the Claude Code harness with 512K context. HLE for some models is the text-only subset. The Hugging Face card lists NL2Repo-Bench as 64.0; the report and change log say 65.4.

• Gemini 3.8 Flash p.5

  Results as of September 2026, as published in Google's model card. Prices in the card list $0.75 / $3.75 with $1.50 / $7.50 as the regular rate. LVBench value for 3.8 Flash is the agentic setting (static: 87.1%).

• Muse Spark 1.3 model page; Muse Spark 1.3 evaluation methodology

  Muse Spark 1.3, GPT 5.6 Sol and Opus 5 at max reasoning effort; Muse Spark 1.2 at xhigh. Meta reports the highest comparable score from its own runs, official leaderboards or the other makers' own reports. OSWorld 2.0 version 08.08, except 1.2 on 06.24. Agentic IF Index is a Meta-internal composite. Meta's table has no MRCR scores for Opus 5.

• DeepSeek-V4-Pro model card (0813)

  Code-agent tasks use DeepSeek Harness minimal mode at max effort, temperature 1.0, top_p 0.95. DSBench rows are DeepSeek internal test sets. Competitor figures as published by DeepSeek.

• Qwen3.8-Max model card (Benchmark Results)

  Qwen's own table. Terminal Bench 2.1 run with Claude Code (avg@10); competitor scores are the best published across harnesses. SWE-bench Pro uses a corrected task set. Rows marked in-house are Qwen benchmarks. Fable 5 results may involve fallbacks. Agents' Last Exam is split into pass rate and score.

• Claude Fable 5 Table 8.1.A, p.251–252

  Rows where the card reports a Fable 5 score. Fable 5 scores include production safeguards and fallback to Opus 4.8, so some are slightly below Mythos 5. Mythos 5 uses adaptive thinking at max effort, averaged over 5 trials. Some Fable rows were run by external evaluators (Surge, Andon Labs, Artificial Analysis, Databricks, Zapier, Harvey). Terminal-Bench 2.1 GPT-5.5 is with Codex CLI, Gemini with Gemini CLI. GDPval-AA Elo as of 6 Jun 2026. Competitor figures from developers' published results or leaderboards. Competitor values for AutomationBench, OfficeQA Pro and Harvey are left blank: they couldn't be matched to columns reliably from the PDF.

• Grok 4.5 §2–7, p.5–19

  Grok 4.5 at high effort; competitors at max (Fable 5 with fallback), GPT-5.5 at xhigh. Results come from mixed harnesses and evaluators; some competitor figures are taken from developers' own cards. CyberGym is run without production safeguards.

• Muse Spark 1.2 launch post charts; Muse Spark 1.2 & Muse Code evaluation methodology

  Values read from Meta's published bar charts. Terminal-Bench 2.1 and DeepSWE v1.1 run each model in its own agent product (Muse Code for 1.2, mini-swe-agent for 1.1, Claude Code, Codex, Grok Build, Antigravity CLI), pass@1 averaged over five attempts, so DeepSWE is not harness-identical to the official leaderboard. Muse Spark at xhigh, Opus 5 and GPT at max, Grok and Gemini at high. The MCP Atlas chart shows Opus 5 at xhigh and omits GPT 5.6 Terra, Grok 4.5 and Gemini 3.6 Flash. Meta Internal Coding Bench is 440 tasks from Meta's own codebase.

• Claude Sonnet 5 Table 8.1.A, p.115–116; SWE-bench variants §8.2, p.116

  Claude Sonnet 5 results use adaptive thinking at max effort, averaged over five trials. BrowseComp is the single-agent score (86.6% multi-agent) with a 10M-token limit and compaction. GPT-5.5 Terminal-Bench 2.1 figure uses Codex CLI. Competitor figures come from the developers' own published results.

• Muse Spark 1.1 model page; report §5

  Highest reasoning effort for every model (high for Gemini 3.1 Pro, max for Opus 4.8). "Muse Spark (1.0)" is the earlier model. Third-party scores are Meta's best-effort runs.

• Claude Opus 4.8 Table 8.1.A, p.194–195

  Opus 4.8 uses adaptive thinking at max effort with default sampling, averaged over 5 trials. Competitor figures come from the developers' own published cards or leaderboards.

• Dola Seed 2.1 Turbo Tables 1, 2, 5, 11 (pp.9, 10, 19, 53)

  ByteDance's own runs from the Seed2.1 model card. Seed 2.1 Pro (not on BytePlus) is shown for reference. ProgramBench is reported as the average pass rate.