Skip to content

Runtime guide

Serve speech with vLLM

Run Oruk ASR v2 on your GPU and send a short recording to its OpenAI-compatible transcription endpoint.

Start a local endpoint

Use a CUDA environment compatible with vLLM. The server downloads the selected weights on first use. Keep the model ID identical in the server and client commands.

GPU server
python -m pip install "vllm[audio]==0.31.0" librosa
vllm serve oruk/orukqwen-asr-1.7b-v2 --max-model-len 2048 --host 127.0.0.1 --port 8000

Send a short recording

From a second terminal on the same host, upload the sample. The JSON response includes the transcript in the text field.

Terminal
curl --fail --location --output sample.wav https://oruk.ai/samples/oruk-quickstart.wav
curl --fail-with-body http://127.0.0.1:8000/v1/audio/transcriptions \
  -F file=@sample.wav \
  -F model=oruk/orukqwen-asr-1.7b-v2 -F language=en

Match the decoder to the evaluation

The v2 model card reports results with the Transformers beam-search configuration. Standard vLLM sampling uses greedy decoding. Evaluate the served configuration on your own audio before comparing it with reported WER.