Macro F1

F1 is the harmonic mean of precision and recall for a single class: precision is the share of predictions for that class that were correct, recall is the share of true instances the model found, and the harmonic mean punishes a model that buys one at the expense of the other. Macro F1 computes that score separately for every class and then averages the results giving each class equal weight, regardless of how many examples it has.

That last detail is the whole point. Emotion corpora are badly imbalanced — neutral typically dominates, and labels such as disgust or embarrassment may be a rounding error of the data. A model that predicts neutral for everything can post a respectable accuracy figure while being useless, because accuracy is dominated by the frequent classes. Macro F1 gives the rare classes the same vote as the common ones, so that model scores near zero on every class it never predicts and its average collapses.

The contrast with micro F1 matters too. Micro F1 pools all predictions before computing the score, which weights classes by frequency and therefore behaves much like accuracy. When you see a single F1 number reported without a qualifier, it is worth finding out which one it is, because on imbalanced emotion data the two can differ by a wide margin for the same model.

Reading the two numbers together is more informative than either alone. On our 64-system benchmark, oruk Spectra records 77.6 percent accuracy with a macro F1 of 0.81. Macro F1 sitting above accuracy indicates the model is not living off the common classes. Compare that with systems further down the table: SpeechBrain fine-tuned on IEMOCAP posts 39.2 percent accuracy but a macro F1 of only 0.211, and an XLS-R model fine-tuned on RAVDESS reaches 23.8 percent accuracy at 0.118 macro F1. Those gaps are the signature of a model that handles two or three classes and guesses at the rest.

Two caveats when comparing published figures. In multilabel settings, where several labels can be true of one clip, F1 is computed per label and then averaged, and the result depends entirely on the decision thresholds — so a macro F1 quoted without its thresholds is not reproducible. And macro F1 says nothing about which errors are costly. If confusing frustration with anger matters more to your application than missing surprise, a single averaged number will not tell you, and you need the per-class breakdown.