aiaimodel.cards
← All benchmarks

Terminal-Bench 3.0

Completing multi-step tasks in a real terminal: shell commands, builds, data wrangling and debugging, graded automatically.

Your workloadRAG over docs · 12K in · 600 out · 50K requests/mo · 30% cached
  1. 1DeepSeek-V4.1-Flash · DeepSeekbest value
    30.0%
  2. 2Grok 4.6 · xAI
    26.0%

“Best value”: no other model here reports a higher score at a lower cost for your workload. Scores are self-reported by each maker with different settings, so treat close calls as ties.

How each maker reported it

  • DeepSeek-V4.1-Flash Table 3, p.33

    All models at max reasoning effort. Code-agent benchmarks use DeepSeek Harness Minimal mode with 1M context; DeepSWE uses mini-SWE; visual agent tasks use the Claude Code harness with 512K context. HLE for some models is the text-only subset. The Hugging Face card lists NL2Repo-Bench as 64.0; the report and change log say 65.4.

  • Grok 4.6 §2–7, p.7–28

    Grok models at high effort (EEBench Grok 4.6 xhigh is 60.0%). Competitors at max effort (Fable 5 with fallback); GPT-5.6 Sol at xhigh on APEX-SWE and CADGenBench, high on DeepSearchQA. Most results come from third-party evaluators, each with its own harness. CyberGym is run without production safeguards.

Only each model's own maker's number is shown; scores a maker reports for rival models are left out.