Skip to content

Try Resonance 2 early.

Tell us a little about yourself and what you’re building.

A brief description of your use case.

We’ll use these details to review your application and contact you about access. Privacy policy

Explore oruk

Python integration

Speech-to-text and emotion in Pipecat

Use pipecat-oruk to bring Oruk Realtime transcripts and phrase-level vocal emotion into a Pipecat pipeline. Interim text arrives while speech continues; final text and expression events can feed your application. The adapter supplies the speech input layer. Your application chooses its own LLM, text-to-speech provider and response behavior.

Integration at a glance

Packagepipecat-oruk==0.1.0rc3; Oruk-maintained community release candidate
Pinned runtimePipecat 1.8.1; hosted adapter tested on Python 3.11–3.14. The example below uses Python 3.12.
Model and transportoruk-realtime (Preview), WebSocket from your Python server
Languages32 transcription locales with automatic language detection; emotion quality needs validation in your setting
OutputInterimTranscriptionFrame, TranscriptionFrame, raw provider events and completed-turn results
File exampleMono PCM16 WAV, 8–96 kHz, at most 300 seconds; paced sending by default
CostUses shared speech understanding minutes. Hobby starts at $9/month with 250 minutes; other plans and trial limits apply.

The package is separate from an upstream Pipecat release. It needs valid API access and an active trial or plan for Realtime. See allowances, overage and spending caps. Cancellation can still consume already processed audio.

Run the file-stream example

This pins the maintained complete Python program, including input checks, event handling and cleanup. It streams a public sample through a real Pipecat pipeline and prints JSON events. Keep ORUK_API_KEY in the server environment; do not put it in browser code or a repository.

Terminal · pinned Pipecat example
git clone https://github.com/Oruk-AI/pipecat-oruk.git
cd pipecat-oruk
git checkout fda0c0067bb6b21c1f5490720f3c1e521ea47d85
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install 'pipecat-oruk==0.1.0rc3' 'pipecat-ai==1.8.1'
curl --fail --location --max-time 30 \
  https://oruk.ai/samples/oruk-quickstart.wav -o sample.wav
# Set ORUK_API_KEY in this server environment, then:
python examples/pipecat_stream_file.py sample.wav --wait-for-emotions

Replace sample.wav with your own supported file once this works. The sample is for integration, not a held-out accuracy evaluation. --wait-for-emotions holds final text until the turn result is complete; interim text remains immediate.

What appears on stdout

Each line is a JSON object. Text, scores, event counts and timing vary; do not compare a run with a hard-coded expected emotion or assume that all words have emotion labels.

typeFields to readMeaning
interimtext, user_idProvisional transcript; later text can revise it.
finaltext, user_id, orukFinal transcript and available Oruk metadata.
provider_eventturn_id, audio_offset, eventOriginal event, including separate phrase-emotion updates.
completeturn_id, audio_offset, resultCompleted turn result, serialized by the example.

The adapter exposes OrukSTTService, OrukEventFrame and OrukTurnCompletedFrame. Keep the turn ID and audio offset when joining asynchronous events. The Realtime event guide explains phrase boundaries and score interpretation.

Try the browser transport

The pinned browser example sends the supplied recording through Pipecat SmallWebRTC to the Python service. It displays received transcripts, phrase events and completion receipts. It runs on localhost and demonstrates speech transport, cancellation and reconnection. It does not use a microphone, LLM or speech synthesis.

Terminal · localhost browser example
# In the same repository and virtual environment:
python -m pip install 'pipecat-oruk[agent]==0.1.0rc3' 'pipecat-ai==1.8.1'
curl --fail --location --max-time 30 https://oruk.ai/samples/oruk-quickstart.wav \
  -o examples/pipecat_browser/sample.wav
curl --fail --location --max-time 30 https://oruk.ai/brand/kit/oruk-primary.png \
  -o examples/pipecat_browser/oruk-primary.png
python examples/pipecat_browser/server.py
# Open http://127.0.0.1:3119/ in your browser.

Use the browser setup and source for the full instructions. This local development server is not a ready-made multi-user deployment. Keep secrets in its environment and add your own application authentication before exposing a service.

Error handling and evidence

The file example checks channel count, sample width, rate and duration before sending. It reports service errors and exits unsuccessfully on invalid input or a failed pipeline. A disconnect after interim text is not a completed turn. Retain request receipts and check account usage before deciding to resend; cancelling a connection does not erase metered audio.

The September 11 recorded demonstration and verification receipts show three completed turns, a cancelled turn and reconnection against the production service. Those receipts establish that integration exercise, not model accuracy, an end-to-end voice agent or assistant barge-in.

Expression estimates may be missing or wrong and do not establish a speaker's feelings or intent. Validate any response policy on representative conversations. For clip-level continuous emotion and speaking-style scores, compare the separate speech emotion API.

Reference: versioned package, pinned source and SDK documentation. Reviewed .