Speech emotion
What is speech emotion recognition?
Speech emotion recognition (SER) is the task of detecting a speaker’s emotional state — anger, happiness, sadness, fear, surprise, disgust, frustration — directly from the sound of their voice. Unlike sentiment analysis on a transcript, SER works on the audio itself, so it hears the difference between a calm complaint and one about to boil over, even when the words are identical.
What emotion recognition models listen to
- Prosody
- Pitch contours, speaking rate, pauses, and stress patterns — the melody of speech that carries most emotional information.
- Voice quality
- Breathiness, tension, jitter, and loudness dynamics that separate genuine warmth from strained politeness.
- Temporal dynamics
- How emotion builds and shifts across turns — frustration rising over a call, not just a snapshot of one utterance.
- Linguistic context
- What was said, fused with how it was said, so a flat "great, thanks" can be flagged as sarcastic-sounding delivery when the tone contradicts the words.
How speech emotion recognition is measured
SER systems are evaluated on labeled corpora such as IEMOCAP and MELD, usually reported as accuracy and weighted or macro F1 across emotion classes. Published numbers are hard to compare because papers use different label sets, splits, and scoring code. oruk maintains speech-emotion-bench to fix that: open models use the full 64,384 held-out clips across ~20 languages and 7 emotion classes; closed/API and audio-LLM systems use a fixed 5,000-clip subset. All 64 rows use the same label mapping and scorer, but not identical audio.
In the July 2026 release, the top measured result is oruk Spectra at 77.6% accuracy and 0.810 macro F1, ahead of the best open model (emotion2vec+ at 68.7%) and far ahead of frontier multimodal APIs like Gemini 3 Flash Preview (46.0%) and GPT-Audio 1.5 (43.3%).
Where speech emotion AI is used
Contact-center teams can score completed calls to surface moments of frustration for human review instead of relying on keyword matches alone. Voice-agent teams can analyze prerecorded evaluation calls to find interactions where the agent’s response should be revised. Health and research teams can review affect labels across recorded sessions while keeping interpretation with qualified people.
oruk exposes calibrated emotion recognition through one API
Emotion is one output of oruk’s speech understanding models, alongside English transcription and a separate 16-label speaking-style task. API v1 accepts prerecorded audio files and can return emotion alone or combine every output in unified analysis.