transcribe

Bot-audio transcription for the eval harness.

When a scenario judges a spoken response, the harness needs the text of what the bot actually said — the transcription of its synthesized audio, not the text fed to the TTS. EvalTranscriber provides that by calling an STT service’s run_stt() directly on the captured audio, mirroring how EvalJudge calls run_inference() — there is no pipeline to run.

This works with any STT whose run_stt(audio) transcribes the buffer and returns: local models like Whisper (which loads in its constructor) and HTTP services. A live/streaming STT (e.g. Deepgram’s WebSocket service) does not fit — its run_stt ships audio to a socket and the results arrive out of band, so nothing is yielded.

The transcriber takes an already-built STTService; EvalTranscriber.from_config() constructs one from a scenario’s judge.transcription: mapping — service (default "moonshine", a local model), model, and optional padding_secs. The escape hatch is transcription.factory: "my_pkg.my_func" — an importable callable taking (config, sample_rate) and returning an STTService. Audio is resampled to 16 kHz before transcription, a rate STT services expect.

class pipecat.evals.transcribe.EvalTranscriber(service: STTService, *, padding_secs: float = 2)

Bases: object

Transcribes bot audio by calling an STT service’s run_stt() directly.

Takes an already-built STTService; from_config() builds one from a scenario’s judge.transcription: mapping. Use as an async context manager:

async with EvalTranscriber.from_config({"model": "base"}) as t:
    text = await t.transcribe(pcm, sample_rate=24000)

Only STTs whose run_stt(audio) transcribes the buffer and returns are supported (local models like Whisper, HTTP services) — not live/streaming ones, whose results arrive out of band.

__init__(service: STTService, *, padding_secs: float = 2)

Initialize the transcriber.

Parameters:

• service – A constructed STTService (e.g. from from_config()).

• padding_secs – Silence padded onto each side of the segment before transcription, giving the STT a clean lead-in/lead-out so it keeps onset/offset words (see SILENCE_PAD_S). 0 disables padding.

classmethod from_config(config: dict | None) → EvalTranscriber

Build an EvalTranscriber from a scenario’s judge.transcription: mapping.

Honors a custom factory (dotted path to a callable taking (config, sample_rate) and returning an STTService); otherwise dispatches on the service name (default "moonshine"). Add providers by extending this. To use a fully custom setup, construct EvalTranscriber directly with your own STTService and pass it to pipecat.evals.harness.EvalSession.from_scenario().

Parameters:

config – transcription mapping, or None for the Moonshine default. An optional padding_secs overrides the silence padding (default SILENCE_PAD_S; see __init__()).

Returns:

A configured EvalTranscriber (not yet started).

Example:

# In the scenario: transcription.factory: "my_pkg.make_stt"
def make_stt(config, sample_rate):
    return WhisperSTTService(...)

async start() → None

Prepare the transcriber. The STT service loads its model on construction.

async transcribe(pcm: bytes, sample_rate: int) → str

Transcribe one audio segment to text.

Calls the STT service’s run_stt() on the resampled, padded audio and joins the TranscriptionFrame``s it yields. Returns ``"" when the audio contains no recognizable speech.

Parameters:

• pcm – Raw 16-bit little-endian mono PCM.

• sample_rate – Sample rate of pcm; resampled to 16 kHz if needed.

Returns:

The transcribed text, stripped, or "" for silence.

async aclose() → None

Release resources. The STT service holds no pipeline-managed state here.