speech

User-audio generation for the eval harness.

When a scenario defines a user_audio: block, the harness generates raw audio for each user turn using that TTS config and streams it to the bot as RTVI raw-audio messages. The bot’s STT then processes it for real, exercising the full input audio path.

EvalSpeech sets up an already-built TTSService with a minimal frame-processor lifecycle (no pipeline or worker) and calls its run_tts directly for each turn, collecting the TTSAudioRawFrame``s. Only local (e.g. Kokoro) or HTTP (e.g. ``CartesiaHttpTTSService) services are supported — websocket-streaming TTS needs a pipeline to manage its connection. Generated audio is cached at <cache-dir>/<sha256>.wav (keyed by cache_key + lower-cased text); the cache file’s actual sample rate is checked on load and regenerated on mismatch, so you can experiment with sample rates without bloating the cache.

EvalSpeech.from_config() constructs the service from a user_audio mapping (dispatch + a user_audio.factory escape hatch) and wraps it; tts_cache_key() and tts_sample_rate() derive the cache identity and rate.

pipecat.evals.speech.tts_sample_rate(voice_cfg: dict) → int

The sample rate a user_audio block asks for (default 16 kHz).

pipecat.evals.speech.tts_cache_key(voice_cfg: dict) → str

A stable identity for a user_audio config, for caching synthesized audio.

Covers the audio’s semantic identity (service, voice, model) but not the sample rate, so different rates reuse the same slot (a mismatch just triggers regeneration in EvalSpeech.generate()).

class pipecat.evals.speech.EvalSpeech(service: TTSService, *, sample_rate: int, cache_key: str, cache_dir: str | Path | None = None, use_cache: bool = True)

Bases: object

Generates user audio for a scenario by calling a TTS service’s run_tts.

Takes an already-built TTSService; from_config() builds one from a scenario’s user_audio mapping. start() sets the service up with a minimal frame-processor lifecycle (no pipeline or worker — just a task manager, a clock, and a StartFrame), then generate() calls run_tts directly and collects the audio. Synthesized audio is cached on disk (keyed by cache_key + text), so re-runs skip synthesis entirely.

Only local (e.g. Kokoro) or HTTP (e.g. CartesiaHttpTTSService) services are supported. Websocket-streaming TTS needs a running pipeline to manage its connection lifecycle, which run_tts alone doesn’t drive, so it is rejected (see __init__()).

Use as an async context manager:

async with EvalSpeech(service, sample_rate=16000, cache_key="...") as speech:
    pcm, sample_rate = await speech.generate("hello world")

__init__(service: TTSService, *, sample_rate: int, cache_key: str, cache_dir: str | Path | None = None, use_cache: bool = True)

Initialize the generator.

Parameters:

• service – A constructed local or HTTP TTSService (e.g. from from_config()).

• sample_rate – Output sample rate the service produces.

• cache_key – Stable identity for the service config (see tts_cache_key()); combined with the text to key the cache.

• cache_dir – Where to store cached audio. Defaults to <user-cache-dir>/pipecat/tts (or $PIPECAT_EVALS_CACHE_DIR).

• use_cache – When False, ignore any cached audio and don’t write new cache files — every utterance is freshly synthesized.

Raises:

ValueError – If service is a websocket-streaming TTS service (run_tts can’t be driven without a pipeline to manage its connection); use a local or HTTP service instead.

property sample_rate: int

Sample rate (Hz) of the audio this generates.

classmethod from_config(voice_cfg: dict, *, cache_dir: str | Path | None = None, use_cache: bool = True) → EvalSpeech

Build an EvalSpeech from a scenario’s user_audio mapping.

Honors a custom factory (dotted path to a callable taking (voice_cfg, sample_rate) and returning a TTSService); otherwise dispatches on the service name. Add providers by extending this. To use a fully custom setup, construct EvalSpeech directly with your own TTSService and pass it to pipecat.evals.harness.EvalSession.from_scenario().

Parameters:

• voice_cfg – user_audio mapping — service (kokoro or cartesia) and voice at minimum; optional sample_rate (default 16 kHz), plus model / api_key (defaults to $CARTESIA_API_KEY) for Cartesia.

• cache_dir – Where to store cached audio (see __init__()).

• use_cache – When False, force fresh synthesis (see __init__()).

Returns:

A configured EvalSpeech (not yet started).

Example:

# In the scenario: user_audio.factory: "my_pkg.make_tts"
def make_tts(voice_cfg, sample_rate):
    return RimeTTSService(...)

async start() → None

Set the service up so generate() can call run_tts directly.

A TTS service normally gets its task manager, clock, and output sample rate from the pipeline that hosts it. Provide a minimal stand-in here (no pipeline, no worker): a StartFrame sets the sample rate and opens any HTTP session the service needs. enable_metrics is off so run_tts never tries to push a MetricsFrame (there is no downstream).

async generate(text: str) → tuple[bytes, int]

Return (pcm, sample_rate) for text — from cache or freshly synthesized.

Call serially: each call runs one run_tts on the shared service and collects its audio.

Parameters:

text – The utterance to synthesize.

Returns:

Tuple of (pcm_bytes, sample_rate) — raw 16-bit little-endian mono PCM.

async aclose() → None

Stop and clean up the TTS service (closes any HTTP session).