ModelCards
← All benchmarks

Earnings-22 keyphrase recall, without context

We haven't written a description for this benchmark yet; see the card notes below for how each maker ran it.

Your workloadRAG over docs · 12K in · 600 out · 50K requests/mo · 30% cached
  1. 1Voxtral Small Transcribe 3 · Mistral AI
    81.4%

“Best value”: no other model here reports a higher score at a lower cost for your workload. Scores are self-reported by each maker with different settings, so treat close calls as ties.

How each maker reported it

  • Voxtral Small Transcribe 3 launch post (charts)

    Mistral's own evaluation. FLEURS: macro-average WER (CER for ja, ko, th, zh) over 45 languages supported by all seven systems compared, language specified (forced) or auto-detected (inferred). Diarization: macro-average cpWER over AMI-IHM, AMI-SDM, Switchboard, CallHome and AISHELL-4 (30-min versions of AMI and AISHELL). Keyphrase recall on Earnings-22 with context bias of key terms plus 0–100 distractors. Artificial Analysis WER Index as charted by Mistral. Lower is better except keyphrase recall.

Only each model's own maker's number is shown; scores a maker reports for rival models are left out.