stt

Together AI speech-to-text service implementation.

This module provides a STT service using Together AI’s WebSocket API:

  • TogetherSTTService: WebSocket-based real-time STT using an OpenAI-compatible realtime transcription endpoint.

class pipecat.services.together.stt.TogetherSTTSettings(model: str | None | _NotGiven = <factory>, extra: dict[str, Any]=<factory>, language: Language | str | None | _NotGiven = <factory>)[source]

Bases: STTSettings

Settings for the Together AI STT service.

class pipecat.services.together.stt.TogetherSTTService(*, api_key: str, sample_rate: int | None = None, base_url: str = 'wss://api.together.ai/v1', settings: TogetherSTTSettings | None = None, ttfs_p99_latency: float = 1.0, **kwargs)[source]

Bases: WebsocketSTTService

Together AI speech-to-text service.

Provides real-time speech recognition using Together AI’s WebSocket API with OpenAI-compatible speech-to-text endpoints.

Example:

stt = TogetherSTTService(
    api_key="your-api-key",
    settings=TogetherSTTService.Settings(
        model="openai/whisper-large-v3",
        language=Language.EN,
    ),
)
Settings

alias of TogetherSTTSettings

__init__(*, api_key: str, sample_rate: int | None = None, base_url: str = 'wss://api.together.ai/v1', settings: TogetherSTTSettings | None = None, ttfs_p99_latency: float = 1.0, **kwargs)[source]

Initialize the Together AI STT service.

Parameters:

  • api_key – Together AI API key for authentication.

  • sample_rate – Audio sample rate in Hz. If None, uses the pipeline’s rate.

  • base_url – The URL of the Together AI WebSocket API.

  • settings – Runtime-updatable settings for the STT service.

  • ttfs_p99_latency – P99 latency from speech end to final transcript in seconds. Override for your deployment. See https://github.com/pipecat-ai/stt-benchmark

  • **kwargs – Additional arguments passed to WebsocketSTTService.

can_generate_metrics() → bool[source]

Check if this service can generate processing metrics.

language_to_service_language(language: Language) → str | None[source]

Convert a Language enum to Together AI language format.

Parameters:

language – The language to convert.

Returns:

The language code string, or None if not supported.

async start(frame: StartFrame)[source]

Start the Together AI STT service.

Parameters:

frame – The start frame containing initialization parameters.

async stop(frame: EndFrame)[source]

Stop the Together AI STT service.

Parameters:

frame – The end frame.

async cancel(frame: CancelFrame)[source]

Cancel the Together AI STT service.

Parameters:

frame – The cancel frame.

async run_stt(audio: bytes) → AsyncGenerator[Frame | None, None][source]

Send audio data to Together AI for transcription.

Parameters:

audio – Raw audio bytes to transcribe.

Yields:

Frame – None (transcription results come via WebSocket callbacks).

async process_frame(frame: Frame, direction: FrameDirection)[source]

Process frames with Together AI-specific handling.

Parameters:

  • frame – The frame to process.

  • direction – The direction of frame flow in the pipeline.

async push_frame(frame: Frame, direction: FrameDirection = FrameDirection.DOWNSTREAM)

Push a frame downstream, tracking TranscriptionFrame timestamps for TTFB.

Stores the timestamp of each TranscriptionFrame for TTFB calculation. If the frame is marked as finalized (via request_finalize/confirm_finalize), reports TTFB immediately and cancels any pending timeout. Otherwise, TTFB is reported after a timeout.

Parameters:

  • frame – The frame to push.

  • direction – The direction to push the frame.

async stop_ttfb_metrics(*, end_time: float | None = None)

Stop time-to-first-byte metrics collection and push results.

Parameters:

end_time – Optional timestamp to use as the end time. If None, uses the current time.