A public score compresses the choices behind a benchmark into a single number. Scripted recordings, conversational turns, the transcript normalizer and the decoding configuration all affect the ranking. An internal estimate also inherits uncertainty about which kinds of speech the held-out board contains.
We assembled two roughly fifty-hour evaluation pools and scored common slices with the same baseline models. The calibration uses leave-one-model-out predictions. Each held-out model is evaluated using a mapping fitted without its own board score.
Twelve models were evaluated. Among the eleven with published AA labels, the Clean-9 proxy reached a Spearman rank correlation of 0.954. Mean absolute error was 0.54 WER percentage points. Holding out an entire lab reduced the rank correlation to 0.909. These results help choose experiments, while close differences still need a board submission.
Sources and experiments
Continue exploring
Check the result against the protocol.
Inspect the dated model versions, sample coverage, and uncertainty in the complete report. Then evaluate the current API on audio representative of your application.