aiaimodel.cards
← All benchmarks

SEC-Bench Pro

We haven't written a description for this benchmark yet; see the card notes below for how each maker ran it.

Your workloadRAG over docs · 12K in · 600 out · 50K requests/mo · 30% cached
  1. 1GPT-6 Astra · OpenAIbest value
    85.4%
  2. 2GPT-6.1 Sol · OpenAIbest value
    78.8%
  3. 3GPT-6 Sol · OpenAI
    66.3%
  4. 4DeepSeek-V4.1-Flash · DeepSeekbest value
    62.8%
  5. 5GPT-6 Luna · OpenAIbest value
    34.2%

“Best value”: no other model here reports a higher score at a lower cost for your workload. Scores are self-reported by each maker with different settings, so treat close calls as ties.

How each maker reported it

  • GPT-6 Astra Launch post (openai.com/index/gpt-6-astra); system card §6, §10.1; GPT-6.1 Sol addendum §5, §9

    Coding and reasoning rows come from the GPT-6 Astra launch post (3 Sep 2026); OpenAI's own comparison table, settings as published there. The system card reports safety and Preparedness evaluations, not general capability benchmarks. Cyber and bio scores are at maximum reasoning effort; bio scores are refusal-adjusted. HealthBench scores are length-adjusted. GPT-6.1 Sol and some GPT-5.6 Sol values come from the GPT-6.1 Sol addendum (29 Sep 2026). Unverified: openai.com blocks automated access, so the coding and reasoning figures were read from copies of the launch post, and they don't appear in the system card or developer docs.

  • GPT-6.1 Sol §5, §9

    The addendum reports safety and Preparedness evaluations, not general capability benchmarks. Cyber and bio scores are at maximum reasoning effort; bio scores are refusal-adjusted. HealthBench scores are length-adjusted.

  • GPT-6 Sol Appendix A, p.120–154

    The card reports safety and Preparedness evaluations, not general capability benchmarks. Cyber scores are at maximum reasoning effort; GPT-5.6 Sol cyber values are its maximum scores using more tokens. HealthBench scores are length-adjusted. SandboxBench is targets solved out of 22. Astra values only where the appendix states them.

  • DeepSeek-V4.1-Flash Table 3, p.33

    All models at max reasoning effort. Code-agent benchmarks use DeepSeek Harness Minimal mode with 1M context; DeepSWE uses mini-SWE; visual agent tasks use the Claude Code harness with 512K context. HLE for some models is the text-only subset. The Hugging Face card lists NL2Repo-Bench as 64.0; the report and change log say 65.4.

  • GPT-6 Luna Appendix A, p.120–154; GPT-6.1 Sol addendum Table 9

    The card reports safety and Preparedness evaluations, not general capability benchmarks. Cyber and bio scores are at maximum reasoning effort; HealthBench scores are length-adjusted. MentalHealthBench comes from the GPT-6.1 Sol addendum.

Only each model's own maker's number is shown; scores a maker reports for rival models are left out.