stt

Soniox speech-to-text service implementation.

class pipecat.services.soniox.stt.SonioxContextGeneralItem(*, key: str, value: str)[source]

Bases: BaseModel

Represents a key-value pair for structured general context information.

key: str

value: str

class pipecat.services.soniox.stt.SonioxContextTranslationTerm(*, source: str, target: str)[source]

Bases: BaseModel

Represents a custom translation mapping for ambiguous or domain-specific terms.

source: str

target: str

class pipecat.services.soniox.stt.SonioxContextObject(*, general: list[SonioxContextGeneralItem] | None = None, text: str | None = None, terms: list[str] | None = None, translation_terms: list[SonioxContextTranslationTerm] | None = None)[source]

Bases: BaseModel

Structured context that steers transcription and translation.

Every section is optional; supply only the ones that are relevant. Soniox caps the whole object at 8k tokens.

Learn more about context in the documentation: https://soniox.com/docs/stt/concepts/context

general: list[SonioxContextGeneralItem] | None

text: str | None

terms: list[str] | None

translation_terms: list[SonioxContextTranslationTerm] | None

class pipecat.services.soniox.stt.SonioxInputParams(*, model: str = 'stt-rt-v5', audio_format: str | None = 'pcm_s16le', num_channels: int | None = 1, language_hints: list[Language] | None = None, language_hints_strict: bool | None = None, context: SonioxContextObject | str | None = None, enable_speaker_diarization: bool | None = False, enable_language_identification: bool | None = False, client_reference_id: str | None = None)[source]

Bases: BaseModel

Real-time transcription settings.

Deprecated since version 0.0.105: Use settings=SonioxSTTService.Settings(...) instead. Will be removed in 2.0.0.

See Soniox WebSocket API documentation for more details: https://soniox.com/docs/speech-to-text/api-reference/websocket-api#configuration-parameters

Parameters:

model – Model to use for transcription.
audio_format – Audio format to use for transcription.
num_channels – Number of channels to use for transcription.
language_hints – List of language hints to use for transcription.
language_hints_strict – If true, strictly enforce language hints (only transcribe in provided languages).
context – Customization for transcription. String for models with context_version 1 and ContextObject for models with context_version 2.
enable_speaker_diarization – Whether to enable speaker diarization. Tokens are annotated with speaker IDs.
enable_language_identification – Whether to enable language identification. Tokens are annotated with language IDs.
client_reference_id – Client reference ID to use for transcription.

pipecat.services.soniox.stt.is_end_token(token: dict) → bool[source]: Determine if a token is an end token.

pipecat.services.soniox.stt.language_to_soniox_language(language: Language) → str[source]

Convert a Pipecat Language to a Soniox language code.

For a list of all supported languages, see: https://soniox.com/docs/speech-to-text/core-concepts/supported-languages

Bases: STTSettings

Settings for SonioxSTTService.

Parameters:

language_hints – List of language hints to use for transcription.
language_hints_strict – If true, strictly enforce language hints.
context – Customization for transcription. Either a SonioxContextObject or a plain string of background text.
enable_speaker_diarization – Whether to enable speaker diarization.
enable_language_identification – Whether to enable language identification.
max_endpoint_delay_ms – Max ms before endpoint detection finalizes the turn (500-3000).
endpoint_sensitivity – Endpoint detection sensitivity (-1.0 to 1.0); higher finalizes sooner.
endpoint_latency_adjustment_level – Reduces endpoint latency vs. the default (0-3); higher finalizes sooner but may reduce accuracy.
client_reference_id – Client reference ID to use for transcription.

The max_endpoint_delay_ms, endpoint_sensitivity and endpoint_latency_adjustment_level settings only take effect when vad_force_turn_endpoint=False; otherwise Soniox endpoint detection is disabled and these settings are ignored.

class pipecat.services.soniox.stt.SonioxSTTService(*, api_key: str, url: str = 'wss://stt-rt.soniox.com/transcribe-websocket', sample_rate: int | None = None, model: str | None = None, audio_format: str = 'pcm_s16le', num_channels: int = 1, params: SonioxInputParams | None = None, vad_force_turn_endpoint: bool = True, should_interrupt: bool = True, settings: SonioxSTTSettings | None = None, ttfs_p99_latency: float | None = 0.35, **kwargs)[source]

Bases: WebsocketSTTService

Speech-to-Text service using Soniox’s WebSocket API.

This service connects to Soniox’s WebSocket API for real-time transcription with support for multiple languages, custom context, speaker diarization, and more.

For complete API documentation, see: https://soniox.com/docs/speech-to-text/api-reference/websocket-api

Settings: alias of SonioxSTTSettings

__init__(*, api_key: str, url: str = 'wss://stt-rt.soniox.com/transcribe-websocket', sample_rate: int | None = None, model: str | None = None, audio_format: str = 'pcm_s16le', num_channels: int = 1, params: SonioxInputParams | None = None, vad_force_turn_endpoint: bool = True, should_interrupt: bool = True, settings: SonioxSTTSettings | None = None, ttfs_p99_latency: float | None = 0.35, **kwargs)[source]

Initialize the Soniox STT service.

Parameters:

api_key – Soniox API key.
url – Soniox WebSocket API URL.
sample_rate – Audio sample rate.
model –
Soniox model to use for transcription.

Deprecated since version 0.0.105: Use settings=SonioxSTTService.Settings(model=...) instead. Will be removed in 2.0.0.
audio_format – Audio format for transcription. Defaults to "pcm_s16le".
num_channels – Number of audio channels. Defaults to 1.
params –
Additional configuration parameters, such as language hints, context and speaker diarization.

Deprecated since version 0.0.105: Use settings=SonioxSTTService.Settings(...) instead. Will be removed in 2.0.0.
vad_force_turn_endpoint – Controls turn detection mode. When True (Pipecat mode, default): Soniox endpoint detection is disabled and a VADUserStoppedSpeakingFrame sends a finalize message to Soniox. When False (Soniox turn detection mode): Soniox endpoint detection is enabled and controls turn endings. Proposes a turn start on the local VAD signal when a VAD analyzer is configured (most responsive) or on the first transcript token otherwise, and a turn stop when the endpoint is detected.
should_interrupt – Whether to interrupt the bot when the user starts speaking in Soniox turn detection mode (vad_force_turn_endpoint=False). Only applies when using Soniox’s built-in endpoint detection. Passed along to the user turn strategies this service recommends, which own the interruption; a user-supplied user_turn_strategies overrides the recommendation and this setting with it. Defaults to True.
settings – Runtime-updatable settings. When provided alongside deprecated parameters, settings values take precedence.
ttfs_p99_latency – P99 latency from speech end to final transcript in seconds. Override for your deployment. See https://github.com/pipecat-ai/stt-benchmark
**kwargs – Additional arguments passed to the STTService.

can_generate_metrics() → bool[source]

Check if this service can generate processing metrics.

Returns:: True, as Soniox STT supports metrics generation.

service_metadata_frame() → STTMetadataFrame[source]

Request external turn strategies in Soniox’s turn-detection mode.

With vad_force_turn_endpoint=False Soniox’s endpoint detection decides turn endings and this service proposes turn boundaries, so the user aggregator resolves those rather than running local VAD/smart-turn. In the default Pipecat mode (vad_force_turn_endpoint=True) the STT proposes no turns, so the defaults are left in place. Applied unless the user passed their own user_turn_strategies.

async setup(setup: FrameProcessorSetup)[source]

Set up the service and connect.

Parameters:: setup – Configuration object containing setup parameters.

async stop(frame: EndFrame)[source]

Stop the Soniox STT websocket connection.

Parameters:: frame – The end frame.

async cancel(frame: CancelFrame)[source]

Cancel the Soniox STT websocket connection.

Compared to stop, this closes the connection without sending the end-of-audio frame.

Parameters:: frame – The cancel frame.

async run_stt(audio: bytes) → AsyncGenerator[Frame | None, None][source]

Send audio data to Soniox STT Service.

Parameters:: audio – Raw audio bytes to transcribe.
Yields:: Frame – None (transcription results come via WebSocket callbacks).

async process_frame(frame: Frame, direction: FrameDirection)[source]

Processes a frame of audio data, either buffering or transcribing it.

Parameters:

frame – The frame to process.
direction – The direction of frame processing.

async push_frame(frame: Frame, direction: FrameDirection = FrameDirection.DOWNSTREAM)

Push a frame downstream, tracking TranscriptionFrame timestamps for TTFB.

Stores the timestamp of each TranscriptionFrame for TTFB calculation. If the frame is marked as finalized (via request_finalize/confirm_finalize), reports TTFB immediately and cancels any pending timeout. Otherwise, TTFB is reported after a timeout.

Parameters:

frame – The frame to push.
direction – The direction to push the frame.

async start_stt_usage_metrics(usage: STTUsage)

Start STT usage metrics collection.

Parameters:: usage – Usage information for the STT operation.

async stop_ttfb_metrics(*, end_time: float | None = None)

Stop time-to-first-byte metrics collection and push results.

Parameters:: end_time – Optional timestamp to use as the end time. If None, uses the current time.