Voice activity detection (VAD)

Voice activity detection is the task of deciding, moment by moment, whether an audio signal contains speech. A VAD consumes a waveform and returns the time ranges where someone is talking, discarding silence, background noise, music, and other non-speech sound. It is normally the first stage of a speech pipeline, because almost everything downstream is either wasted or actively misled when run on audio containing no speech.

The simplest approach thresholds short-term energy: frames louder than an adaptive noise floor are marked as speech. It is cheap and works acceptably in quiet conditions, but it fails whenever the noise is loud or speech-like, marking door slams and music as speech while dropping quiet or breathy talk. Statistical VADs, of which the WebRTC implementation is the most widely deployed, model speech and noise as separate distributions across frequency bands and decide with a likelihood ratio, which is more robust. Neural VADs such as Silero and the segmentation models in pyannote classify short frames with a small trained network and hold up considerably better under real noise, at higher computational cost.

Whatever the classifier, a VAD needs smoothing to be usable. Raw frame-level decisions flicker, because natural speech contains stop closures and brief pauses that genuinely contain no sound. Implementations therefore add a hangover period that keeps the speech state alive for a short window after the last positive frame, plus minimum-duration constraints that drop implausibly short segments. Tuning those parameters is most of the practical work: too aggressive and words get clipped at boundaries, too permissive and long silences are absorbed into speech regions.

Performance is a trade-off between two error types, and a single accuracy number hides it. False alarms pass non-speech through as speech, wasting downstream compute and producing output for audio nobody spoke. Misses discard real speech, which is unrecoverable — no later stage can transcribe or score audio the VAD threw away. Which error is worse depends on the application, so the operating point should be chosen deliberately rather than left at a default.

For emotion and speaking-style scoring, VAD does something more specific than saving compute: it keeps the model from labelling nothing. A classifier handed thirty seconds of hold music or room tone will still return scores, and those scores are meaningless. Segmenting first means time-local labels attach to spans where someone was actually speaking, which is what makes a per-segment view of a long recording interpretable.

VAD is often confused with speaker diarization, which is a separate and harder problem. VAD answers whether anyone is speaking; diarization answers who is speaking and when. A VAD provides useful input to diarization but does not perform it, and the oruk API does not offer speaker identification or diarization.