Teaching a speech model what counts as wrong

A transcript can be almost right and still teach a speech model the wrong lesson.

Suppose a model gets an entire sentence right except for one name. Compared with the other answers it sampled, that transcript might receive a positive advantage. Apply that positive learning signal to every token and the wrong name shares in the credit.

That was our starting point for reinforcement learning in speech recognition. We wanted to tell the model which parts of an answer deserved encouragement. We eventually released OrukQwen, an open-weights post-training of Qwen3-ASR. But the experiments after that release exposed a harder question: which differences between a recording and a reference transcript are mistakes in the first place?

A person says, “I, I think we should go.” One transcript keeps the repetition. Another writes, “I think we should go.” A system can move closer to either reference without hearing the speaker any better. Once we started optimizing those small decisions, they could dominate the gains we were trying to measure.

Give credit to the words that earned it

We used Group Sequence Policy Optimization, or GSPO, to sample 64 transcripts for each training clip and compare their rewards. A transcript's advantage tells us how it performed relative to its sampled alternatives. Our token-credit variant uses word-level Levenshtein alignment with the reference to decide where a positive advantage should apply.

When an answer has a positive advantage, tokens belonging to mistaken words do not receive that positive credit. A deleted word has no output token to mark, so its mask falls on the following token or the end-of-sequence token. The method still relies on the reference being a useful definition of correctness. It cannot know whether an apparent error is a legitimate alternative spelling or a reference that silently edited the speech.

The first Qwen campaign compared three recipes: baseline GSPO, GSPO with that token-credit mask, and a Dr. GRPO-style variant of advantage and length normalization. All three improved the initial internal English evaluation by roughly one absolute WER point. The masked arm won at the final 400-update checkpoint, but the differences between arms on the pooled English aggregate were not significant when we averaged the late checkpoints. That is useful evidence for RL post-training; it is weaker evidence for declaring one credit rule the winner.

We had already seen that distinction in earlier Orukeet experiments. A September counterfactual-credit run improved AMI WER from 10.208% to 10.127%, while a supervised control reached 9.962%. In a later matched three-seed comparison, counterfactual credit did not consistently improve on its supervised initializer. The Qwen recipe was a new experiment, not a continuation of a proven universal advantage for token-level credit.

The model we released is halfway back to the base

The public release came from the masked arm, followed by two weight-averaging steps. First we averaged three related RL continuations into one set of weights, a model soup. Then we blended that soup halfway back toward the original Qwen model, using the principle behind WiSE-FT.

The continuations also differed in credit handling. The original controlled masked arm left negative advantages unchanged; one member of the released soup additionally withheld negative credit from correct tokens. The released result therefore evaluates this combined lineage rather than a clean ablation of positive-credit masking alone.

This mattered more than the name “soup” suggests. The unblended RL soup scored 4.95% mean WER across nine public English test sets. The base scored 4.94%. Blending the soup 50/50 with the base brought the mean to 4.757%. The complete recipe worked even though the intermediate soup did not improve the aggregate.

The released model, oruk/orukqwen-asr-1.7b, has Apache-2.0 weights and the same architecture, tokenizer and inference interface as its Qwen base. The family name says 1.7B; the full model contains about 2.04 billion parameters. Averaging happens before deployment, so the download is one model rather than an ensemble that runs several recognizers for every recording.

The original paired harness evaluation reduced mean WER from 4.936% to 4.757%, about a 3.6% relative reduction. A later H200 reproduction rounded to the same 4.76% mean, with small per-set differences; the figures here retain the original campaign measurements so the comparisons stay consistent. These are our runs of the official harness, not an independently assigned leaderboard rank.

The release's training lineage used 14.26 A100-hours, or 14.81 including a crashed attempt, plus at most 7.5 A100-hours for data selection. The initial study through release used 87 logged A100-hours. Training for each lineage run used one A100 at a time; the wider campaign ran parallel experiments. Those numbers describe different scopes and neither is the cost of all the later work.

More training can improve the wrong thing

We then expanded the pool from about 55 hours to roughly 476 hours, trained many continuations and tried additional soups. The selected larger-data candidate was worse than the public release by 0.037 percentage points on the nine-set mean. Its paired 95% bootstrap interval was entirely above zero.

Other sensible ideas also failed to transfer. Consensus labels from several recognizers looked better on one development set and significantly worse on another, largely through deletions. Prompts asking for “verbatim” or “clean” transcription did not give us reliable control. Training only on difficult clips made all seven candidates in that comparison worse. Opening more of the audio encoder to training did not produce a dependable weights-only gain in the grid we tested.

The recurring failure was transcription style. Some references preserve repeated words and discourse markers; others edit them away. A model trained against one convention can lose on the other even when the underlying speech content is unchanged. Scoring rules add another layer: the harness removes common fillers such as “uh” and “um,” so a conspicuous change in filler production may contribute little to its WER.

Our long-form diagnostic made the trade-off visible. A verbatim-focused continuation reduced WER against the verbatim Earnings-21 reference from the base's 7.72% to 7.44%. Score those same outputs against a more heavily edited reference and the ordering reverses: 8.60% for the base, 9.20% for the continuation. The recording did not change. The definition of the desired transcript did.

Making a feature invisible to the reward did not keep it fixed. When we removed style distinctions from content scoring, repetitions and acronym spacing could still drift toward the training data's conventions. The reward had stopped measuring a behavior that the model was still changing.

Hold the style still while improving the content

The later recipe gave those two objectives separate targets. Content was scored against the human reference after a style-neutral filter. A second term penalized a mismatch between the candidate's style units and the released model's own transcript of that same clip.

In compact form: reward = content score − λ × style mismatch. The mismatch counts changes in units such as fillers, partial words, immediate repetitions, discourse markers and letter-spaced acronyms. Tokens that only carry style receive the pin term's advantage; content tokens receive the combined signal. This anchors a clip to an existing behavior rather than imposing “always remove repetitions” or “always preserve them” across every recording.

The strongest measured candidate averaged eight runs drawn from two pin strengths. It reached 4.661% mean WER with the experimental beam-search configuration. This candidate is still private. Its headline improvement over the public release splits almost evenly between two changes: decoding reduced the mean from 4.757% to 4.708%; the new weights then reduced it to 4.661% at that same decoding.

That last weights-only improvement was 0.047 percentage points, with a 95% paired bootstrap interval of −0.090 to +0.003 on the nine-set mean. The interval includes zero. On the seven current public sets the measured difference was −0.062 points, with the reported interval just excluding zero. These are exploratory campaign comparisons after repeated evaluation and proxy calibration, not a pristine final test of an untouched recipe. The union of the two pin strengths was also chosen after inspecting their development results.

The remaining trade-offs matter. Beam search made the candidate about three times slower than the public release's greedy decoding in the A100 measurements. Its English configuration also bans CJK-script tokens; that guard is inappropriate as a default for multilingual transcription. SPGISpeech remained worse than the original base. A lower aggregate did not eliminate the conflict between transcript conventions.

Did the gain travel to different recordings?

A subsequent follow-up compared the private candidate with v1 at the same beam decoding on 11,080 clip-condition records from four corpora: EasyCom, VOiCES, SLUE-VoxCeleb and Nigerian English from OpenSLR 70. EasyCom contributed both glasses-microphone and close-talk conditions for the same speech, so these are five evaluation conditions rather than five independent corpora.

The follow-up used utterance-capped WER, which caps an utterance's errors at its reference length to limit the influence of runaway output. Its equal-condition mean fell from 10.44% to 10.27%. The strongest change was in EasyCom's glasses-microphone recordings: 28.25% to 27.47%. VOiCES and SLUE-VoxCeleb were effectively unchanged; Nigerian English rose from 5.57% to 5.70%.

What changed inside the model

We also examined the base, public release and private candidate directly. About 98% of the squared weight change was in the decoder. That number partly reflects the training setup: most encoder layers were frozen. More informative interventions swapped encoders between models and replaced decoder activations layer by layer. In the sampled diagnostics, encoder swaps moved WER by at most about 0.15 points, while decoder patching recovered the output difference gradually across depth.

The evidence points to substantial changes in how the model writes down speech. It does not establish that RL cannot improve acoustic recognition. The diagnostics are small, and a nearly unchanged average attention weight on audio is not a complete test of how audio is used.

There was a surprisingly tangible feature in the decoder updates: a direction associated with filler tokens. Steering along it changed how often the model wrote fillers. It barely moved WER near the unsteered setting, and it did not control repetitions in the same way. Since the harness removes many fillers, the most visually obvious internal feature was not a ready-made explanation for the benchmark gain.

Precision complicated the interpretation too. Rounding the private candidate's reconstructed FP32 soup to BF16 changed 29 of 200 diagnostic transcripts; the corresponding v1 comparison changed one. Some apparent geometry between the tiny weight updates could be reproduced by a rounding-only control. We did not run the full harness on the FP32 candidate. At this scale, the saved numerical representation is part of the experimental condition.

The failures changed how we measure progress

This work spans more than the successful release. Earlier transducer experiments taught us to separate improving on an initializer from beating continued supervised training. The YODAS and LibriSpeech credit experiments never established a reliable recognition gain from the aligned mask. Their best small development improvement still had an interval crossing zero.

Even correctness checks needed care. One early transducer pilot was invalidated when BF16 batch shape changed replayed action log-probabilities by about 0.0076. We cached the sampled frozen logits and recomputed trainable residuals in FP32 rather than loosening the check. In another audit, two transcripts changed when the decoding batch size changed even though total word errors stayed identical.

The adjacent weight-editing work supplied a different warning. One later edit removed 334 pooled word errors relative to its P1 comparator across public test data while making the equal-corpus macro WER worse. Large datasets dominate pooled counts; a macro mean gives every dataset the same weight. Both calculations can be correct. They answer different questions. Those edits were training-free follow-ups to RL models, not additional RL training successes.

The practical lesson is to keep content errors, transcript convention, decoding and aggregation visible as separate decisions. For our next experiments, that means evaluating new weights with the same decoder, retaining stronger reference-style controls and keeping FP32 masters through averaging. It also means treating a small selected gain differently from a result that survives a fresh evaluation.

The public OrukQwen weights are available to try today. The later candidate is an experiment we are still evaluating. The question we want to answer on a new recording is concrete: did the model recover more of what was said while preserving the kind of transcript the application needs?

Methods and reproducibility

The original Qwen campaign used the Open ASR harness at commit edd684247365. Its nine-public-set score is an equal-weight mean of dataset WERs, not WER pooled across every word. The seven-public-set mean excludes LibriSpeech clean and other; neither is interchangeable with a leaderboard aggregate that includes private evaluation sets. All public-set results in the first figure use the original A100 campaign rather than mixing in the later H200 reproduction.

Campaign paired intervals use 2,000 bootstrap resamples, with Earnings22 resampled by merged session. They reflect the recorded evaluation populations, not independent repetitions of the entire model-selection campaign, and are not corrected for adaptive selection. The later fresh-data follow-up uses capped WER and 10,000 paired resamples; its shared EasyCom speech and secondary-comparison status remain relevant to interpretation.

The v1 training pool contained 20,000 English clips totaling 54.6 hours. The initial run used 11,304 clips before continuations sampled further from the pool. The later data expansions and pinning experiments are separate phases.

The accompanying eight-second ribbon loop illustrates positive credit separating into word-aligned strands, with the mistaken strand losing that credit while its geometry stays intact. It is a conceptual illustration, not measured gradients, model activity or a corrected transcript. The figures use retained aggregate measurements. Our retained source pack records the inspected local, Conductor and Hugging Face revisions as of November 10, 2026.

Research trail

Sources

• OrukQwen public weights and model card, inspected at revision 121f0fd6e8990971d9ffc0c1fb036afa0af29a3a. The initial release weights and training lineage are documented in its provenance file; the November 6 card adds the H200 reproduction.
• Qwen3-ASR base model. The pretrained recognizer used for the public release.
• GSPO. Sequence-level policy optimization used as the foundation for the Qwen RL runs.
• Model soups. Prior work on averaging related fine-tuned models into one weight set.
• WiSE-FT. Prior work on interpolating a fine-tuned model with its pretrained base.
• Open ASR Leaderboard harness. Evaluation implementation used in the campaign.

The later experiments draw on Oruk's November 8 technical report and mechanistic study, the private November 9 candidate card, the September transducer results and the retained credit-assignment and weight-editing audits. These have not all been publicly released; v1 is the public model available to test.