Resonance-2: hear how it was said

“It’s eleven o’clock” can be a reminder, a complaint, or a reason to celebrate. A transcript preserves the words. A voice carries the difference.

Today we’re introducing Resonance-2, Oruk’s next speech emotion and speaking-style model. It reads audio directly and returns continuous scores across 31 categories, from frustration and relief to warmth, hesitation, and sarcasm. It gives voice applications a second channel of information alongside speech recognition.

The release improves recognition on our conversational evaluation slice and leads the systems in our fixed, 546-clip acted-emotion diagnostic. Below, you can hear real examples, inspect the results, and make the same API call yourself.

Same words. Six deliveries.

A single recording can sound angry, irritated, and disappointed at once. Resonance-2 keeps those signals separate. The continuous scores let an application follow changes in expression without reducing every turn to one emotion word.

Most categories can coexist. Six pairs instead share a signed axis. Sad and happy occupy opposite ends of one axis; hesitant and confident occupy another. Each axis ranges from −1 to +1, so its two ends cannot both appear in the selected output.

For example, sad__happy: −0.8 becomes a sad score of 0.8 and a happy score of zero. This is a dominance score: the difference between the two calibrated outputs. It does not mean an 80% probability that someone is sad. A value near zero means neither end dominates; the underlying signals can also cancel.

The other 19 categories retain independent scores between zero and one. Scores do not sum to one. For applications that need discrete labels, the API also returns a thresholded list. The f1 regime is the default; precision uses more conservative thresholds. Both return the same continuous scores, and either can return an empty label list.

Human listeners often agree on a recording’s broad character while choosing different words for it. Collapsing their ratings into one winning label throws away that structure. Resonance-2 uses annotation fractions: if three of four listeners select “worried,” its training target is 0.75.

We combine those dense targets with other emotion-annotated speech corpora. When a corpus labels emotions but does not annotate speaking style, the missing style labels contribute no loss. An unasked question should not become a negative answer.

The retained model trained for three passes over 386,658 rows, including 19,666 Oruk human-annotated clips with 32,441 accepted ratings. We kept the inherited speech encoder fixed and updated the affect classifier. Validation selected a blend of 75% updated and 25% original classifier weights, preserving more of the existing model’s behavior.

We ran Resonance-2 through the public API on 546 CREMA-D recordings: 91 actors, six intended emotions, and the same sentence in every clip. The fixed words make this a useful test of vocal delivery. Resonance-2 classified 337 recordings correctly, for 61.7% accuracy, compared with 55.7% for our previous Fourier model on these same recordings.

The 95% interval for Resonance-2’s accuracy is 58.1–65.9%, resampling whole actors. The scorer ranks selected emotion labels before mapping them into the benchmark’s six classes. All 170 empty or out-of-taxonomy answers count as errors.

This comparison measures recognition of intended, acted emotion in one sentence. Training overlap has not been audited, so it is not an independent held-out leaderboard. It also does not establish that the model outperforms human listeners.

We also compared the new classifier with original Resonance across all 31 categories. Here the metric is macro F1, which gives each category equal weight. Clips are held out from this fine-tuning run by connected recording group, with thresholds fixed on a separate calibration split.

The clearest human-data gain is on conversation: +5.80 percentage points, with a paired group-bootstrap interval of +4.29 to +7.17. The overall human-data gain is small and its interval includes zero. The newer-rating check is essentially unchanged. Conversation and newer ratings are subsets of the full human evaluation, not additional independent test sets.

These are development-era holdouts that have been reused; exposure during older pretraining is not certified. They support a more specific conclusion than “better everywhere”: the retained update improves this conversational slice and the other-corpus evaluation, while leaving substantial work on natural, ambiguously expressed emotion.

Evaluation definitions, counts, and uncertainty.

Send audio to Resonance-2 alongside your existing speech recognizer. Use its scores as context for how your application responds: a signal to slow down, ask a clarifying question, or leave room for a person to speak. The endpoint returns clip-level affect; your speech recognizer continues to provide the transcript.

curl --fail-with-body 'https://speech-api.oruk.ai/v1/audio/resonance-2?regime=f1' \
  -H "Authorization: Bearer $ORUK_API_KEY" \
  -H "X-Request-ID: $(uuidgen)" \
  -F model=oruk-resonance-2 \
  -F file=@clip.wav

The response includes scores for all 31 labels, six signed axes, 19 independent unipolar scores, and selected labels. It also returns the model and calibration revisions so you can record exactly what produced a result. Clips can be 0.1–120 seconds and up to 30 MiB; WAV, FLAC, MP3, M4A, OGG, and WebM are supported.

On our A100 worker, five-second clips took 24.2 ms median worker request time at concurrency one. The public route includes upload, decoding, and network overhead: a separate four-second upload test measured 1.41 seconds median end to end. Measure that full path for your application.

Resonance-2 is available in preview with existing Oruk API keys, at the same price as Resonance and within the same plan allowance. Original Resonance remains available. You do not need a new subscription or add-on.

The model does not require a transcript or language code, but accuracy varies with language and recording conditions. Treat the scores as estimates of perceived vocal expression, and evaluate them on the speech your application will actually hear.

Open the Resonance-2 API documentation for the full response schema, threshold options, and retry behavior.