Speech emotion
What is speech emotion recognition?
Speech emotion recognition (SER) is the task of detecting a speaker’s emotional state — anger, happiness, sadness, fear, surprise, disgust, frustration — directly from the sound of their voice. Unlike sentiment analysis on a transcript, SER works on the audio itself, so it hears the difference between a calm complaint and one about to boil over, even when the words are identical.
What emotion recognition models listen to
- Prosody
- Pitch contours, speaking rate, pauses, and stress patterns — the melody of speech that carries most emotional information.
- Voice quality
- Breathiness, tension, jitter, and loudness dynamics that separate genuine warmth from strained politeness.
- Temporal dynamics
- How emotion builds and shifts across turns — frustration rising over a call, not just a snapshot of one utterance.
- Linguistic context
- What was said, fused with how it was said, so "great, thanks" reads as sarcasm when the tone contradicts the words.
How speech emotion recognition is measured
SER systems are evaluated on labeled corpora such as IEMOCAP and MELD, usually reported as accuracy and weighted or macro F1 across emotion classes. Published numbers are hard to compare because papers use different label sets, splits, and scoring code. Oruk maintains speech-emotion-bench to fix that: 64,384 held-out clips across ~20 languages and 7 emotion classes, scored with one identical pipeline for 27+ systems — open models, frontier APIs, and Oruk’s own.
On that protocol, Oruk Spectra leads at 77.6% accuracy and 0.810 macro F1, ahead of the best open model (emotion2vec+ at 68.7%) and far ahead of frontier multimodal APIs like Gemini 3 Flash Preview (46.0%) and GPT-Audio 1.5 (43.3%).
Where speech emotion AI is used
Voice agents use emotion recognition to de-escalate before a customer churns. Contact centers score calls on real frustration instead of keyword guesswork. Health and research teams track affect over time. And AI assistants use it to respond in kind — matching energy, slowing down, or escalating to a human when tone demands it.
Oruk exposes calibrated emotion recognition through one API
Emotion is one output of Oruk’s speech understanding models, alongside English transcription and a separate 16-label speaking-style task. API v1 accepts prerecorded audio files and can return emotion alone or combine every output in unified analysis.