Dataset guide
Speech emotion recognition datasets compared
The best dataset is the one that matches the claim you need to make. A model can score highly on clean acted clips and still fail on customer calls, overlapping dialogue, unfamiliar accents, or microphones it never saw during training.
Quick comparison
| Dataset | Speech | Context | Best use | Main caveat |
|---|---|---|---|---|
| IEMOCAP | Acted and improvised dyadic sessions | Dialogue, audio, video, motion capture | Widely reported four-class SER benchmark | Ten actors; speaker and session leakage can inflate results |
| MELD | Scripted television conversations | Multi-party dialogue with text, audio, and video | Conversational and multimodal emotion | Television acting, laughter tracks, and show-specific acoustics |
| RAVDESS | Studio-recorded acted speech and song | Isolated clips with audio and video | Clean, balanced controlled experiments | Professional acting and a small set of fixed statements |
| CREMA-D | Acted sentences from 91 performers | Isolated audio and video clips with crowd ratings | Speaker diversity and perceptual annotations | Acted delivery; evaluator demographics affect labels |
| MSP-Podcast | Naturalistic podcast speech | Long-form, varied speakers and recording conditions | Closer to in-the-wild deployment audio | Label ambiguity, class imbalance, and access restrictions |
For a research baseline
IEMOCAP remains useful because many papers report the same four-class task: angry, happy or excited, sad, and neutral. It is not a universal measure of emotion understanding. Report the exact class mapping, session split, speaker split, metric, and whether happy and excited were merged.
For deployment evidence
Prefer naturalistic, domain-matched recordings and a speaker-independent holdout. If the application analyzes calls, test real call codecs, silence, crosstalk, background noise, and the accents you expect. Public benchmarks should be a reproducible baseline, not a substitute for that evaluation.
Five rules that prevent misleading results
- 01
Split by speaker. A speaker must never appear in both training and test data. Otherwise the model can exploit identity and recording conditions.
- 02
Keep the native label uncertainty. When several annotators disagree, retain the distribution or agreement score instead of pretending there is one certain inner state.
- 03
Report macro F1 or UAR. Plain accuracy can look good when neutral or another common class dominates the dataset.
- 04
Run a cross-corpus test. Training and testing on different corpora reveals whether the model learned emotion-related patterns or dataset-specific shortcuts.
- 05
Publish exclusions and failures. List corrupted clips, API errors, label mappings, usable sample counts, confidence intervals, and the exact model snapshot.
See those rules applied
The oruk benchmark publishes its label mapping, per-system sample counts, confidence intervals, protocol caveats, downloadable results, and separate IEMOCAP and MUStARD snapshots.