services

Service constructors for the eval harness: a Pipecat service from a scenario’s config mapping.

The dispatchers pick a provider by its service: name, or call a factory; the per-provider builders do the rest. The built-in names are the local, keyless services: Ollama for the judge and the persona, Kokoro for the user’s voice, Moonshine and Whisper for the bot’s audio. Any other provider is a factory: a dotted path to a callable that takes the config mapping and returns the service, so a scenario stays a description of the test while the provider, its settings, and its key live in code:

# my_evals.py
def judge(config):
    return OpenAILLMService(settings=OpenAILLMService.Settings(model=config["model"]))

def voice(config):
    return CartesiaHttpTTSService(
        api_key=os.environ["CARTESIA_API_KEY"],
        settings=CartesiaHttpTTSService.Settings(voice=config["voice"]),
    )

# in the scenario
judge:
  eval: {factory: my_evals.judge, model: gpt-4o-mini}
user:
  speech: {factory: my_evals.voice, voice: 71a7ad14-091c-4e8e-a314-022ece01c121}

What a factory must return: the judge’s and the persona’s LLM must be OpenAI-compatible, a BaseOpenAILLMService subclass; the user’s TTS must be a local or HTTP service, since the cache around it drives run_tts directly and a WebSocket-streaming one yields nothing there; the bot’s STT can be any pipeline STT. Provider imports stay inside the functions, so importing this module is cheap.

pipecat.evals.services.stt_service_from_config(config: dict | None) → STTService[source]

Build the STT that transcribes the bot’s audio, from a judge.transcription: mapping.

A factory (a dotted path to a callable taking the config) builds it; otherwise the service name picks one, moonshine by default. Any pipeline STT works; the pipeline sets its sample rate.

Parameters:

config – The transcription mapping, or None for the Moonshine default.

Returns:

A constructed STTService (model loaded), ready to add to the pipeline.

pipecat.evals.services.tts_service_from_config(voice_cfg: dict, *, cache_dir: str | None = None, use_cache: bool = True) → CachingTTSService[source]

Build the user-audio TTS, wrapped in a cache, from a user_audio mapping.

A factory (a dotted path to a callable taking the voice config) builds the inner service; otherwise the service name picks one, kokoro. The wrapper synthesizes each user utterance once and reuses it across runs. The pipeline sets the sample rate.

Parameters:
  • voice_cfg –

    user_audio mapping — service or factory, and voice, at minimum. service: cartesia still builds a Cartesia HTTP TTS, with a warning.

    Deprecated since version 1.9.0: Use factory instead of service: cartesia. Will be removed in 2.0.0.

  • cache_dir – Where to store cached audio (see CachingTTSService).

  • use_cache – When False, force fresh synthesis.

Returns:

A configured CachingTTSService (not yet started).

pipecat.evals.services.kokoro_service(voice_cfg: dict) → TTSService[source]

Build a local Kokoro TTS service from the user_audio config.

No API key and no per-run cost; the model is downloaded once and cached under ~/.cache/pipecat/kokoro-onnx.

Parameters:

voice_cfg –

The user.speech config mapping:

  • voice: Kokoro voice id (e.g. af_heart).

  • language: Optional language code (e.g. zh) or Language. When omitted, Kokoro keeps its own default (English). Voices are language-specific, so a non-English language needs a matching voice — af_heart speaks US English whatever the language is set to.

  • speed: Optional speech rate multiplier (Kokoro accepts 0.5 to 2.0; 1.0 is its default). Pauses scale with it too, so a faster rate also shortens the silences at commas.

pipecat.evals.services.cartesia_service(voice_cfg: dict) → TTSService[source]

Build a Cartesia TTS service from the user_audio config.

Deprecated since version 1.9.0: Use user.speech.factory instead: a factory that constructs CartesiaHttpTTSService. Will be removed in 2.0.0.

Parameters:

voice_cfg –

The user.speech config mapping:

  • voice: Cartesia voice id.

  • model: Optional model (defaults to sonic-2).

  • api_key: Optional key (falls back to $CARTESIA_API_KEY).

  • language: Optional language code (e.g. zh) or Language. When omitted, Cartesia keeps its own default (English).

Raises:

RuntimeError – If no API key is given in the config or the environment.

pipecat.evals.services.whisper_service(config: dict) → STTService[source]

Build a local Whisper STT service from the bot_audio config.

It runs on the CPU by default, leaving the GPU to the judge and the audio models; transcription is off the hot path, so the latency is fine. Set device: cuda (and compute_type) in the transcription config for GPU.

Whisper’s non-speech filter is disabled: it only ever hears the bot’s own speech, and the default threshold drops correct transcriptions of TTS speech run to run.

Parameters:

config –

The judge.transcription config mapping:

  • device: cpu (default) or cuda.

  • compute_type: Whisper compute type (int8 on CPU).

  • model: Optional Whisper model (left unset to use Whisper’s own). Whisper’s default is English-only, as is every .en model, so a non-English language needs a multilingual model here (e.g. large-v3-turbo).

  • language: Optional language code (e.g. es) or Language. When omitted, Whisper keeps its own default (English) — it does not auto-detect, so a non-English bot needs this set.

Raises:

ValueError – If language names a language the chosen model can’t transcribe (raised by WhisperSTTService).

pipecat.evals.services.moonshine_service(config: dict) → STTService[source]

Build a local Moonshine STT service from the bot_audio config.

Small and fast on the CPU, and steadier than Whisper on the short bot-answer segments the harness transcribes.

Parameters:

config –

The judge.transcription config mapping:

  • model: Optional architecture, as a Model or the equivalent string (default Model.SMALL_STREAMING). Only base has non-English models, so a non-English language needs it, and raises at construction naming the models it does have if the pairing has none.

  • language: Optional language code (e.g. es) or Language. When omitted, Moonshine keeps its own default (English).

Prefer whisper_service() for a non-English bot: Moonshine’s non-English models transcribe synthesized speech unreliably, returning an empty transcript or dropping the tail of an utterance, and an empty transcript is indistinguishable from a bot that said nothing.

pipecat.evals.services.classifier_from_config(config: dict | None, *, where: str) → BaseClassifier[source]

Build the classifier a judge.eval: block names.

A factory (a dotted path to a callable taking the config) builds it, and may return a BaseClassifier or an LLM service, which an LLMClassifier then asks. Otherwise the block names an LLM, as llm_service_from_config() reads it, and an LLMClassifier asks that.

Parameters:
  • config – Mapping with a factory, or the keys llm_service_from_config() reads. None uses all defaults.

  • where – The config block’s name in the file, for error messages.

Returns:

The configured classifier.

Raises:

ValueError – If service is unknown, factory is not a dotted path, or the factory returned neither a classifier nor an LLM service.

pipecat.evals.services.llm_service_from_config(config: dict | None, *, where: str) → LLMService[Any][source]

Build an LLM service from a service: block, the judge’s or a persona’s.

A factory (a dotted path to a callable taking the config) builds it; otherwise the service name picks a provider, ollama, the default.

Parameters:
  • config –

    Mapping with keys service (default "ollama"), model, optional endpoint, and an optional extra mapping forwarded to the model as top-level request parameters. None uses all defaults. service: openai still builds an OpenAI service, with a warning.

    Deprecated since version 1.9.0: Use factory instead of service: openai. Will be removed in 2.0.0.

  • where – The config block’s name in the file, for error messages ("judge.eval", "simulator").

Returns:

The configured LLM service.

Raises:

ValueError – If service is unknown or factory is not a dotted path.

pipecat.evals.services.ollama_service(config: dict) → LLMService[Any][source]

Build a local Ollama LLM service from the judge: config.

An extra: mapping goes to the model as request parameters, which is how reasoning_effort: none reaches a thinking model. The default model comes with the extras it needs; a named model gets only the extras asked for.

pipecat.evals.services.openai_service(config: dict) → LLMService[Any][source]

Build an OpenAI LLM service from the judge: config.

Deprecated since version 1.9.0: Use judge.eval.factory (or simulator.factory) instead: a factory that constructs OpenAILLMService. Will be removed in 2.0.0.

An extra: mapping is forwarded verbatim as top-level request parameters.