Skip to content

Developer documentation

Voice agents

Build a voice agent with oruk Realtime: stream caller audio, then pass live transcripts and phrase-level emotion scores to your language model.

Oruk does the listening in a voice agent. It transcribes the caller in 32 locales and scores how each phrase sounded. The language model, the speech synthesis and the audio transport are yours. Oruk does not generate speech or run the conversation.

What you build

What Oruk provides in a voice agent and what you supply
PartProvided by
Transcription and toneOruk Realtime
Reasoning and toolsYour language model
Speech synthesisYour provider
Turn-taking and interruptionsYour framework, for example Pipecat or LiveKit Agents
Audio transportYour WebRTC or telephony provider

1. Open a session

Connect from your server so the API key stays there. A session lasts up to ten minutes.

Node.js
import WebSocket from "ws"

const socket = new WebSocket(
  "wss://speech-api.oruk.ai/v1/realtime?model=oruk-realtime",
  "oruk-realtime",
  { headers: { Authorization: `Bearer ${process.env.ORUK_API_KEY}` } },
)

socket.on("open", () => socket.send(JSON.stringify({
  type: "session.update",
  session: { model: "oruk-realtime", language: "auto", sample_rate: 16000, phrase_emotions: true, phrase_silence_ms: 600 },
})))

// Mono signed 16-bit PCM at 16 kHz, as binary frames.
export const sendAudio = (chunk) => socket.send(chunk, { binary: true })

2. Join words and tone

Transcript events and emotion events arrive separately. Final text comes in conversation.item.input_audio_transcription.completed; the phrase’s scores come later in conversation.item.input_audio_emotion.completed. Match them by phrase_id and timestamps.

Node.js
const turns = new Map() // phrase_id -> { text, voice }

socket.on("message", (data) => {
  const event = JSON.parse(data.toString())
  if (event.type === "conversation.item.input_audio_emotion.completed") {
    // Arrives after the words. Attach it to the phrase it belongs to.
    turns.set(event.phrase_id, { text: event.text, voice: event })
  }
  if (event.type === "conversation.item.input_audio_emotion.failed") {
    // Carry on with the text alone. Do not substitute "neutral".
  }
})

3. Decide whether to wait

A phrase ends after 600 ms of silence by default; set phrase_silence_ms between 200 and 2,000. You can send the turn to your language model on the final transcript and add tone to the next turn, or wait for the emotion event and include it now. Waiting costs latency. Measure both on your own calls.

4. Tell the model what the scores mean

Scores run from 0 to 1 independently and describe how the phrase sounded. They are not a reading of what the person feels, and they can be wrong. Say so in the system prompt, and have the agent ask when words and voice disagree.

Related

Voice agent overview · Pipecat · LiveKit · Twilio · Complete phrase-emotion example · Realtime protocol reference