aiaimodel.cards
← All benchmarks

CyberGym

We haven't written a description for this benchmark yet; see the card notes below for how each maker ran it.

Your workloadRAG over docs · 12K in · 600 out · 50K requests/mo · 30% cached
  1. 1DeepSeek-V4.1-Flash · DeepSeekbest value
    88.1%
  2. 2DeepSeek-V4-Pro · DeepSeek
    83.3%
  3. 3Dola Seed 2.1 Turbo · ByteDance
    67.0%

“Best value”: no other model here reports a higher score at a lower cost for your workload. Scores are self-reported by each maker with different settings, so treat close calls as ties.

How each maker reported it

  • DeepSeek-V4.1-Flash Table 3, p.33

    All models at max reasoning effort. Code-agent benchmarks use DeepSeek Harness Minimal mode with 1M context; DeepSWE uses mini-SWE; visual agent tasks use the Claude Code harness with 512K context. HLE for some models is the text-only subset. The Hugging Face card lists NL2Repo-Bench as 64.0; the report and change log say 65.4.

  • DeepSeek-V4-Pro model card (0813)

    Code-agent tasks use DeepSeek Harness minimal mode at max effort, temperature 1.0, top_p 0.95. DSBench rows are DeepSeek internal test sets. Competitor figures as published by DeepSeek.

  • Dola Seed 2.1 Turbo Tables 1, 2, 5, 11 (pp.9, 10, 19, 53)

    ByteDance's own runs from the Seed2.1 model card. Seed 2.1 Pro (not on BytePlus) is shown for reference. ProgramBench is reported as the average pass rate.

Only each model's own maker's number is shown; scores a maker reports for rival models are left out.