Word error rate (WER)

Word error rate measures transcription accuracy as an edit distance. Align the system output against a reference transcript, count the substitutions, deletions and insertions needed to turn one into the other, sum them, and divide by the number of words in the reference. Written out, WER = (S + D + I) / N. A ten-word reference transcribed with one wrong word and one dropped word gives (1 + 1 + 0) / 10, or 20 percent.

Two properties surprise people. WER is not bounded at 100 percent, because insertions are counted against a denominator that does not include them — a system that hallucinates a long passage over a short utterance can score several hundred percent. And WER is not symmetric with accuracy: "97 percent accurate" is marketing for a 3 percent WER, but the two are only equivalent when insertions are rare.

The larger problem with published WER figures is text normalization. Before scoring, both transcripts are usually normalized — lowercased, stripped of punctuation, with numbers, currency, contractions and filler words rewritten to a canonical form. Those choices change the result substantially. Whether "twenty-five dollars" and "$25" count as a match, whether "um" is scored or discarded, and whether "cannot" matches "can not" are all normalization decisions, and vendors do not make them identically. A WER comparison across providers is only meaningful when every system is scored through one normalizer, on the same audio.

WER also weights every word the same. Missing "not" in "I would not recommend this" costs exactly as much as missing "the", though one inverts the meaning and the other does nothing. For that reason WER is often reported alongside character error rate, which is more informative for morphologically rich languages and for names, or alongside task-level metrics such as whether an extracted entity was correct.

For emotion work, WER matters mainly as a bound on what transcript-based analysis can achieve, and it is easy to overrate its importance. Even a perfect transcript discards how something was said. On our benchmark a text-only model reading transcripts reaches 41.2 percent accuracy on seven-class emotion, against 77.6 percent for a model that scores the audio. Driving WER to zero would not close that gap, because the missing information was never in the words.