Verbatim vs. edited ASR normalization
A verbatim transcript preserves repetitions and filled pauses. An edited transcript may remove them for readability. Word error rate compares a hypothesis with a particular reference, so a recognizer can hear a word correctly and still incur an insertion.
Updated
Listen to the recording
Listen for the repeated “what” in the question. The saved model transcript below keeps it. An editor preparing meeting minutes might write “What happened?”
“I’m sorry. The price of food has gone up so much. I mean… It really has. What what happened? I don’t know. Like, honestly, I don’t know what happened.”
Audio from Open Yap 1K by The Agentic Data Company, licensed under CC BY 4.0. Oruk excerpted and downmixed this recording. The transcript is a saved model output. Recording credit and changes.
One question, two scoring conventions
After lowercasing and removing punctuation, “what what happened” matches a three-word verbatim reference. Against the two-word edited reference “what happened,” the same output has one insertion in two reference words, or 50% WER for this question. Capitalization alone cannot reconcile the two.
The paired comparison in Oruk’s OrukQwen-ASR-1.7B model card reports 8.32% to 7.92% WER on AMI-Cleaned and 2.58% to 2.75% on SPGISpeech, from base to trained model. The authors identify preserved repetitions and false starts as a weakness on edited financial references. These figures refer to the paired comparison in the card. They do not measure performance on this grocery clip.
Try the comparison
Save the raw hypothesis. Before scoring, document how each reference handles fillers, repetitions, numbers and punctuation. Score against the matching reference policy and inspect individual insertions. A readability edit can help captions while losing precisely the hesitation a conversation researcher needs.
Read the paired ASR evaluation and its normalization notes
Try it on your own audio.
The demo needs no account. The trial needs a card.