Realtime transcription

wss://api.kittenml.com/v1/realtime?model=kittenasr-enhanced-preview

Use realtime STT when the words should appear while someone is still speaking. This is where KittenASR Enhanced moves beyond vanilla ASR: partial text arrives live, and the final event can preserve expression, pauses, and emphasis.

One WebSocket connection is one realtime session. Use a key with the asr:transcribe permission. Authentication runs once when the connection starts, not for every audio append.

Connect

Send the API key in the WebSocket upgrade request:

Authorization: Bearer sk_kitten_live_...

This header works in server-side WebSocket libraries. The browser's native WebSocket constructor cannot set an Authorization header, so browser apps need an authenticated backend relay that keeps the permanent key private.

For a browser microphone, use WebRTC transcription instead. The backend mints a short-lived client token, then microphone media travels by WebRTC rather than through a WebSocket relay.

The canonical route is /v1/realtime, so the official OpenAI Python client can construct the correct connection URL from the KittenML base URL. KittenML supports realtime transcription events, not the full OpenAI multimodal Realtime feature set. The older /v1/stt/realtime route remains available as a compatibility alias.

Connect with the OpenAI Python SDK

import os
from openai import AsyncOpenAI

client = AsyncOpenAI(
    api_key=os.environ["KITTENML_API_KEY"],
    base_url="https://api.kittenml.com/v1",
)

# The SDK appends /realtime to base_url and opens the WebSocket.
async with client.realtime.connect(
    model="kittenasr-enhanced-preview",
) as connection:
    # Configure first, then append paced audio while reading events concurrently.
    ...

No custom WebSocket URL is required: the SDK appends /realtime to the base URL. A complete tested client is available in the examples repository.

Audio formats

The recommended format is headerless mono PCM16 at 24 kHz.

Format	Sample rate	Session format value
Signed PCM16 little-endian	16 or 24 kHz	audio/pcm, pcm16, or s16le
Float32 little-endian	16 or 24 kHz	pcm_f32le or f32le
G.711 µ-law	8 kHz	g711_ulaw or audio/pcmu
G.711 A-law	8 kHz	g711_alaw or audio/pcma

FLAC, WAV, MP3, and other file containers are not raw realtime audio. Decode them before appending bytes, or use the upload endpoint.

Send 100–200 ms client appends for responsive live audio; the runnable examples use 200 ms. Client append size and transcript update cadence are separate: one append does not guarantee one transcript event.

Think of appends as feeding the stream, not asking the model to speak after every bite.

Session lifecycle

1. Connect and wait for session.created.
2. Send session.update and wait for session.updated.
3. Send base64 audio in input_audio_buffer.append events.
4. Read transcript deltas while audio continues.
5. Send input_audio_buffer.commit when the manual turn is finished.
6. Use conversation.item.input_audio_transcription.completed as the final transcript.
7. Continue with another turn, or send session.close and close the socket.

input_audio_buffer.commit finalizes the current turn; it does not close the WebSocket.

Configure the session

{
  "type": "session.update",
  "session": {
    "type": "transcription",
    "audio": {
      "input": {
        "format": {"type": "audio/pcm", "rate": 24000},
        "transcription": {
          "model": "kittenasr-enhanced-preview",
          "delay": "medium"
        },
        "turn_detection": null
      }
    }
  }
}

transcription.delay selects the target cadence for partial model decoding. It is separate from the recommended 100–200 ms transport append size:

Value	Target partial cadence
minimal	500 ms
low	750 ms
medium	1,000 ms (default)
high	1,500 ms
xhigh	2,000 ms

Set turn_detection to null for explicit commits. For automatic speech turns, send a complete update such as:

{
  "type": "session.update",
  "session": {
    "type": "transcription",
    "audio": {
      "input": {
        "format": {"type": "audio/pcm", "rate": 24000},
        "transcription": {"model": "kittenasr-enhanced-preview"},
        "turn_detection": {
          "type": "server_vad",
          "threshold": 0.3,
          "prefix_padding_ms": 350,
          "silence_duration_ms": 700
        }
      }
    }
  }
}

Understand server VAD settings

These settings apply only when turn_detection.type is server_vad. The defaults shown below are used when you enable server_vad and omit an individual setting.

Setting	Default	Allowed range	If you increase it	If you decrease it
threshold	0.3	0.01–1.0	Makes detection less sensitive. It rejects more background noise but may miss quiet speech.	Makes detection more sensitive. It detects quieter speech but may mistake noise for speech.
prefix_padding_ms	350 ms	0–5,000 ms	Keeps more audio from before speech was detected, protecting initial syllables but including more leading silence or noise.	Keeps less preceding audio, but increases the chance of clipping the beginning of the first word.
silence_duration_ms	700 ms	100–10,000 ms	Allows longer pauses within one turn, reducing accidental splits but delaying the final transcript.	Finalizes turns faster, but may split a sentence when the speaker pauses briefly.

In practical terms, threshold controls speech-detection sensitivity, prefix_padding_ms prevents the beginning of speech from being clipped, and silence_duration_ms determines how long the server waits after speech stops before automatically committing the turn. With the default silence_duration_ms, endpoint detection contributes approximately 700 ms of waiting after the speaker becomes silent, in addition to network and inference time.

Start with the defaults. Raise threshold in a noisy environment, lower it for consistently quiet speakers, and adjust silence_duration_ms only when you need faster finalization or greater tolerance for pauses.

Append audio and commit

{"type":"input_audio_buffer.append","audio":"<base64 audio bytes>"}

{"type":"input_audio_buffer.commit"}

Python example

Install websockets and python-dotenv:

python3 -m pip install websockets python-dotenv

The complete, runnable client is in the KittenML examples repository. It sends paced 200 ms PCM appends while reading transcript events concurrently.

JavaScript example

Install ws:

npm install ws

See the complete Node.js realtime client.

Client events

Event	Purpose
session.update	Configure model, audio, language, and turn detection
input_audio_buffer.append	Append base64-encoded audio
input_audio_buffer.commit	Finalize the current manual turn
input_audio_buffer.clear	Discard uncommitted buffered audio
session.close	End the session
ping	Check the connection

Server events

Event	Purpose
session.created	Session is allocated
session.updated	Configuration was accepted
conversation.item.input_audio_transcription.delta	Provisional transcript snapshot
input_audio_buffer.speech_started	Server VAD detected the start of speech
input_audio_buffer.speech_stopped	Server VAD detected the end of speech
input_audio_buffer.committed	A turn was committed
input_audio_buffer.cleared	Uncommitted audio was discarded
conversation.item.input_audio_transcription.completed	Authoritative final transcript
pong	Response to a client ping
session.closed	Session transport ended; this is not proof that transcription completed
error	The request or session failed

See Realtime events for complete event fields. See Language hints for accepted language values.

Limits

• Realtime and upload STT share five active ASR slots per organization.
• A realtime session holds one slot until it closes.
• Realtime sessions have no fixed duration limit. Keep the connection open only while it is useful and close it explicitly when finished.
• A decoded append may be at most 15 MiB; a WebSocket message may be at most 24 MiB.

See Rate limits and timeouts for 429 handling.