aiaimodel.cards
← All benchmarks

SWE-bench Multimodal

Fixing real issues in real open-source repositories; a patch counts only if the project's own tests pass. Variants differ in task set and languages.

Your workloadRAG over docs · 12K in · 600 out · 50K requests/mo · 30% cached
  1. 1Claude Opus 5.5 · Anthropicbest value
    61.4%
  2. 2Claude Opus 5 · Anthropic
    59.4%
  3. 3Claude Fable 5.1 · Anthropic
    54.7%
  4. 4Claude Sonnet 5.5 · Anthropicbest value
    54.3%
  5. 5Claude Opus 4.8 · Anthropic
    38.4%
  6. 6Claude Opus 4.7 · Anthropic
    34.5%
  7. 7Claude Haiku 5.5 · Anthropicbest value
    30.7%
  8. 8Claude Sonnet 5 · Anthropic
    28.1%

“Best value”: no other model here reports a higher score at a lower cost for your workload. Scores are self-reported by each maker with different settings, so treat close calls as ties.

How each maker reported it

  • Claude Opus 5.5 Table 8.1.A, p.174

    Claude results use adaptive thinking at max effort, averaged over five trials (Terminal-Bench 4.0 at xhigh). Competitor figures come from the developers' own published results.

  • Claude Opus 5 Table 8.1.A, p.148–149

    Claude Opus 5 results use adaptive thinking at max effort, averaged over five trials (ARC-AGI-3 at high effort). GPT-5.6 Sol's FrontierBench figure uses Codex and its ARC-AGI-1 figure is at xhigh. The Fable 5 HealthBench Professional figure carries a footnote reading "Mythos 5". Competitor figures come from the developers' own published results or leaderboards.

  • Claude Fable 5.1 Table 8.1.A, p.167

    Fable 5.1 and Mythos 5.1 share weights, so column 0 covers both (likewise Fable 5 / Mythos 5). Claude results use adaptive thinking at max effort, averaged over five trials. Terminal-Bench 4.0 values are from §8.6, p.171 (Mythos 5.1: 60.9%; GPT-5.6 Sol from the public leaderboard). Competitor figures come from the developers' own published results or leaderboards.

  • Claude Sonnet 5.5 Table 8.1.A, p.109

    Claude results use adaptive thinking at max effort with default sampling, averaged over five trials. Competitor figures come from the developers' own published system cards or leaderboards.

  • Claude Opus 4.8 Table 8.1.A, p.194–195

    Opus 4.8 uses adaptive thinking at max effort with default sampling, averaged over 5 trials. Competitor figures come from the developers' own published cards or leaderboards.

  • Claude Opus 4.7 Table 8.1.A, p.191–192

    Opus 4.7 uses adaptive thinking at max effort with default sampling, averaged over 5 trials (Terminal-Bench 2.0 with thinking disabled; Finance Agent at high effort). Competitor figures come from the developers' own published cards or leaderboards.

  • Claude Haiku 5.5 Table 8.1.A, p.111

    Haiku 5.5 results use adaptive thinking at max effort with default sampling, averaged over five trials. FrontierCode is shown at max effort (Sonnet 5.5 scores 52.1 at xhigh). OSWorld is the offline subset; GPT-6 Luna was run by Anthropic on the same tasks. Other competitor figures come from the developers' own published results.

  • Claude Sonnet 5 Table 8.1.A, p.115–116; SWE-bench variants §8.2, p.116

    Claude Sonnet 5 results use adaptive thinking at max effort, averaged over five trials. BrowseComp is the single-agent score (86.6% multi-agent) with a 10M-token limit and compaction. GPT-5.5 Terminal-Bench 2.1 figure uses Codex CLI. Competitor figures come from the developers' own published results.

Only each model's own maker's number is shown; scores a maker reports for rival models are left out.