Skip to content

Prosody vs. text sentiment in voice agents

Text sentiment describes the attitude expressed by words. Prosody is audible in the timing, stress and pitch of those words. Two callers can give an agent exactly the same transcript and sound quite different.

Updated

Listen to the recording

Play the takes without looking at their labels. Which word receives the most stress? Listen again for a change in pitch or emphasis on “exactly.”

Take A · CREMA-D actor 1005

“That is exactly what happened.”

Take B · CREMA-D actor 1015

“That is exactly what happened.”

Acted audio from CREMA-D. The recording catalog identifies the source files. Database license and contents license.

What the transcript loses

Give a text-only classifier the isolated sentence twice. Its input contains no evidence of which take was played. The actors performed happy and angry readings, respectively. Because both the speaker and the delivery change, this pair illustrates what goes missing without isolating the effect of pitch, timing or speaker identity.

In a voice agent, keep the words alongside their time-aligned audio. Use delivery cues to choose a clarifying question when a literal response seems out of place. A bright reading of an unhappy sentence may be a joke, politeness or a mismatch in the model. A sarcasm label alone cannot decide which.

Try the comparison

For a controlled test, ask the same consenting speaker to record a phrase in several ways. Randomize the playback order. Compare listeners who get the words only with listeners who hear the originals, and record where they disagree.

Compare the saved scores for the two takes

Try the voice emotion analyzer on your own recording

Explore the other paralinguistic signals.

Try it on your own audio.

The demo needs no account. The trial needs a card.