RAVDESS dataset

RAVDESS, released by Livingstone and Russo in 2018, is a validated audiovisual corpus of emotional speech and song from 24 professional actors, twelve female and twelve male. The full release contains 7,356 files across speech and song in audio-only, video-only and audiovisual formats. The speech portion covers eight categories — neutral, calm, happy, sad, angry, fearful, disgust and surprised — with every non-neutral emotion recorded at two intensity levels.

Two features make it convenient to work with. Actors speak only two lexically matched statements, "Kids are talking by the door" and "Dogs are sitting by the door", so word content is held constant across every recording. And the filenames encode the full condition as a fixed sequence of numeric fields covering modality, emotion, intensity, statement, repetition and actor, which means the corpus can be loaded and stratified without a separate label file. It is released under a Creative Commons licence, which is part of why it appears in so many tutorials.

The same features that make it convenient make it narrow. Twenty-four speakers is a small pool, and two sentences give a model very little lexical variation to ignore, so a network can fit the speakers and the sentences rather than the emotional content. The emotions are portrayed on cue by trained actors in a studio, which produces displays far cleaner and more exaggerated than spontaneous speech. Ratings from the corpus validation study confirm that even human listeners do not identify these portrayals perfectly, which puts a ceiling on what any model should be expected to reach.

Our benchmark shows the transfer problem directly. An XLS-R model fine-tuned on RAVDESS records 23.8 percent accuracy and a macro F1 of 0.118 when evaluated on our held-out set of 64,384 clips across roughly twenty languages and seven emotion classes. That is close to the bottom of a field of 64 systems. A comparable pattern appears for SpeechBrain fine-tuned on IEMOCAP, at 39.2 percent accuracy and 0.211 macro F1. Both models do well on the corpus they were tuned on and fall apart elsewhere.

The practical reading is that a high RAVDESS score is weak evidence about deployment performance. It remains a reasonable dataset for teaching, for controlled experiments where lexical content must be held constant, and for sanity checks. It is not a benchmark to select a production system on, and vendor claims resting on it deserve a follow-up question about held-out evaluation.