Twilio calls to Oruk Realtime
Twilio Media Streams sends a phone call’s audio to your server as 8 kHz mu-law over a WebSocket. Oruk Realtime takes 16 kHz 16-bit PCM. This guide shows the server in between: it decodes and resamples each chunk, forwards it, and receives a live transcript and an emotion score for each phrase the caller speaks.
Updated
How the audio travels
| Step | Format | Who does it |
|---|---|---|
| Caller to Twilio | Phone network or SIP | Twilio |
| Twilio to your server | 8 kHz mu-law, base64 in JSON, over WebSocket | Twilio Media Streams |
| Convert | 8 kHz mu-law to 16 kHz 16-bit PCM | Your server |
| Your server to Oruk | Binary PCM frames over WebSocket | Your server |
| Oruk to your server | Transcript and phrase emotion events | Oruk Realtime |
1. Start the stream
Return this TwiML when the call connects.
<Response>
<Start>
<Stream url="wss://your-server.example/twilio" track="inbound_track" />
</Start>
<Say>Go ahead, I'm listening.</Say>
<Pause length="60" />
</Response>2. Bridge the audio
Node.js 22 with ws. The Oruk key stays on this server, and the server accepts only connections Twilio signed.
import { createHmac, timingSafeEqual } from "node:crypto"
import WebSocket, { WebSocketServer } from "ws"
// TWILIO_STREAM_URL is the <Stream url> exactly as written in the TwiML.
const { ORUK_API_KEY, TWILIO_AUTH_TOKEN, TWILIO_STREAM_URL } = process.env
if (!ORUK_API_KEY || !TWILIO_AUTH_TOKEN || !TWILIO_STREAM_URL) throw new Error("Set ORUK_API_KEY, TWILIO_AUTH_TOKEN and TWILIO_STREAM_URL")
// Twilio signs the stream URL with your auth token. Refuse every other client:
// without this check, anyone who finds the address can transcribe on your Oruk key.
function isFromTwilio(signature, streamUrl, authToken) {
const sent = Buffer.from(String(signature ?? ""))
// Twilio may sign the handshake URL with a trailing slash.
return [streamUrl, streamUrl + "/"].some((url) => {
const expected = Buffer.from(createHmac("sha1", authToken).update(url).digest("base64"))
return sent.length === expected.length && timingSafeEqual(sent, expected)
})
}
// G.711 mu-law byte to a signed 16-bit sample.
function mulawToPcm16(byte) {
const u = ~byte & 0xff
const magnitude = (((u & 0x0f) << 3) + 0x84 << ((u >> 4) & 7)) - 0x84
return u & 0x80 ? -magnitude : magnitude
}
// 8 kHz mu-law in, 16 kHz PCM16 out. Each sample is followed by the midpoint to the next.
// `previous` is the last sample of the chunk before, so chunks join without a click.
function mulaw8kToPcm16k(payload, previous = 0) {
const out = Buffer.alloc(payload.length * 4)
let last = previous
for (let i = 0; i < payload.length; i++) {
const sample = mulawToPcm16(payload[i])
out.writeInt16LE((last + sample) >> 1, i * 4)
out.writeInt16LE(sample, i * 4 + 2)
last = sample
}
return { pcm: out, last }
}
new WebSocketServer({
port: 8080,
path: "/twilio",
verifyClient: ({ req }) => isFromTwilio(req.headers["x-twilio-signature"], TWILIO_STREAM_URL, TWILIO_AUTH_TOKEN),
}).on("connection", (twilio) => {
const oruk = new WebSocket(
"wss://speech-api.oruk.ai/v1/realtime?model=oruk-realtime",
"oruk-realtime",
{ headers: { Authorization: `Bearer ${ORUK_API_KEY}` } },
)
const waiting = []
let ready = false, last = 0
oruk.on("open", () => {
oruk.send(JSON.stringify({
type: "session.update",
session: { model: "oruk-realtime", language: "auto", sample_rate: 16000, phrase_emotions: true },
}))
ready = true
for (const chunk of waiting.splice(0)) oruk.send(chunk, { binary: true })
})
twilio.on("message", (raw) => {
const message = JSON.parse(raw.toString())
if (message.event === "media") {
const converted = mulaw8kToPcm16k(Buffer.from(message.media.payload, "base64"), last)
last = converted.last
if (ready) oruk.send(converted.pcm, { binary: true })
else waiting.push(converted.pcm)
}
if (message.event === "stop" && ready) oruk.send(JSON.stringify({ type: "input_audio_buffer.commit" }))
})
oruk.on("message", (data) => {
const event = JSON.parse(data.toString())
if (event.type === "conversation.item.input_audio_transcription.completed") console.log("caller said", event)
if (event.type === "conversation.item.input_audio_emotion.completed") console.log("caller sounded", event)
})
twilio.on("close", () => oruk.close())
oruk.on("error", (error) => { console.error(error.message); twilio.close() })
})Reference code, written against Twilio’s and Oruk’s published protocols. The audio conversion and the signature check are covered by tests in Oruk’s repository; run the whole bridge on your own calls before production.
Questions
- Does this work for calls arriving over a SIP trunk?
- Yes. Media Streams is attached in TwiML, so it works the same whether the call reached Twilio from the phone network or from your SIP trunk.
- Does Oruk connect to SIP directly?
- No. Oruk receives audio over a WebSocket. Twilio, or another telephony provider, terminates the call.
- Will 8 kHz audio lower accuracy?
- Telephone audio carries less detail than a 16 kHz microphone, and upsampling does not add any back. Test on recordings of your own calls before relying on a result.
- Can I transcribe both sides of the call?
- Yes. Set the stream’s track to both_tracks and open one Realtime session for each track, so the two speakers are never mixed.
- What about calls longer than ten minutes?
- A Realtime session lasts up to ten minutes. Open a new session before the limit and carry on forwarding audio.
Try it on your own audio.
The demo needs no account. The trial needs a card.