aiaimodel.cards
← All benchmarks

τ²-bench Retail

Customer-service style tasks where the model uses tools and follows policies over a conversation with a simulated user.

Your workloadRAG over docs · 12K in · 600 out · 50K requests/mo · 30% cached
  1. 1Claude Opus 4.6 · Anthropicbest value
    91.9%
  2. 2Claude Sonnet 4.6 · Anthropicbest value
    91.7%
  3. 3Seed 2.0 Lite · ByteDancebest value
    90.9%
  4. 4Seed 2.0 Pro · ByteDance
    90.4%
  5. 5Claude Opus 4.5 · Anthropic
    88.9%
  6. 6gpt-oss-120b · OpenAIbest value
    67.8%
  7. 7gpt-oss-20b · OpenAIbest value
    54.8%

“Best value”: no other model here reports a higher score at a lower cost for your workload. Scores are self-reported by each maker with different settings, so treat close calls as ties.

How each maker reported it

  • Claude Opus 4.6 Table 2.3.A, p.18

    Opus 4.6 results average 5 trials (SWE-bench: 25) with adaptive thinking at max effort and default sampling. MCP-Atlas is the max-effort score (62.7% at high). Gemini 3 Pro ARC-AGI-2 is Deep Thinking.

  • Claude Sonnet 4.6 Table 2.1.A, p.15

    Claude Sonnet 4.6 results are averaged over 10 trials (SWE-bench over 25) with adaptive thinking at max effort and default sampling; Terminal-Bench 2.0 at default thinking. ARC-AGI-2 is 60.4% at high effort. Competitor columns as reported in the card.

  • Seed 2.0 Lite Tables 4, 8, 12 (pp.10, 13, 17)

    ByteDance's own runs from the Seed2.0 model card, which predates the 260428 snapshot served now. Codeforces is an Elo rating. Seed 2.0 Mini is shown for comparison (same table).

  • Seed 2.0 Pro Tables 3, 8, 11 (pp.9, 13, 16)

    ByteDance's own runs. Competitor columns are as printed in the Seed2.0 card (frontier models available in early 2026). Codeforces is an Elo rating. HLE-text under Agentic is with search tools.

  • Claude Opus 4.5 Table 2.3.A, p.19

    Average of 5 trials with a 64k thinking budget, 200k context, default (high) effort and default sampling. SWE-bench, τ²-bench and MCP Atlas are without extended thinking; Terminal-bench uses a 128k budget (57.8% at 64k). GPT-5.1 MMMU is from the MMMU leaderboard.

  • gpt-oss-120b Figures 1, 2, 4; Table 2

    OpenAI's own pass@1 results at reasoning level high. AIME and HLE with tools; Codeforces with a terminal tool. o4-mini and o3 values read from OpenAI's charts.

  • gpt-oss-20b Figures 1, 2, 4; Table 2

    OpenAI's own pass@1 results at reasoning level high. AIME and HLE with tools; Codeforces with a terminal tool. o3-mini was run without tools on AIME, HLE and Codeforces, so those cells are left empty. Baseline values read from OpenAI's charts.

Only each model's own maker's number is shown; scores a maker reports for rival models are left out.