services
Service constructors for the eval harness: a Pipecat service from a scenario’s config mapping.
The dispatchers pick a provider by its service: name, or call a
factory; the per-provider builders do the rest. The built-in names are
the local, keyless services: Ollama for the judge and the persona, Kokoro for
the user’s voice, Moonshine and Whisper for the bot’s audio. Any other
provider is a factory: a dotted path to a callable that takes the config
mapping and returns the service, so a scenario stays a description of the
test while the provider, its settings, and its key live in code:
# my_evals.py
def judge(config):
return OpenAILLMService(settings=OpenAILLMService.Settings(model=config["model"]))
def voice(config):
return CartesiaHttpTTSService(
api_key=os.environ["CARTESIA_API_KEY"],
settings=CartesiaHttpTTSService.Settings(voice=config["voice"]),
)
# in the scenario
judge:
eval: {factory: my_evals.judge, model: gpt-4o-mini}
user:
speech: {factory: my_evals.voice, voice: 71a7ad14-091c-4e8e-a314-022ece01c121}
What a factory must return: the judge’s and the persona’s LLM must be
OpenAI-compatible, a BaseOpenAILLMService subclass; the user’s TTS must
be a local or HTTP service, since the cache around it drives run_tts
directly and a WebSocket-streaming one yields nothing there; the bot’s STT
can be any pipeline STT. Provider imports stay inside the functions, so
importing this module is cheap.
- pipecat.evals.services.stt_service_from_config(config: dict | None) STTService[source]
Build the STT that transcribes the bot’s audio, from a
judge.transcription:mapping.A
factory(a dotted path to a callable taking the config) builds it; otherwise theservicename picks one,moonshineby default. Any pipeline STT works; the pipeline sets its sample rate.- Parameters:
config – The
transcriptionmapping, orNonefor the Moonshine default.- Returns:
A constructed
STTService(model loaded), ready to add to the pipeline.
- pipecat.evals.services.tts_service_from_config(voice_cfg: dict, *, cache_dir: str | None = None, use_cache: bool = True) CachingTTSService[source]
Build the user-audio TTS, wrapped in a cache, from a
user_audiomapping.A
factory(a dotted path to a callable taking the voice config) builds the inner service; otherwise theservicename picks one,kokoro. The wrapper synthesizes each user utterance once and reuses it across runs. The pipeline sets the sample rate.- Parameters:
voice_cfg –
user_audiomapping —serviceorfactory, andvoice, at minimum.service: cartesiastill builds a Cartesia HTTP TTS, with a warning.Deprecated since version 1.9.0: Use
factoryinstead ofservice: cartesia. Will be removed in 2.0.0.cache_dir – Where to store cached audio (see
CachingTTSService).use_cache – When False, force fresh synthesis.
- Returns:
A configured
CachingTTSService(not yet started).
- pipecat.evals.services.kokoro_service(voice_cfg: dict) TTSService[source]
Build a local Kokoro TTS service from the
user_audioconfig.No API key and no per-run cost; the model is downloaded once and cached under
~/.cache/pipecat/kokoro-onnx.- Parameters:
voice_cfg –
The
user.speechconfig mapping:voice: Kokoro voice id (e.g.af_heart).language: Optional language code (e.g.zh) orLanguage. When omitted, Kokoro keeps its own default (English). Voices are language-specific, so a non-English language needs a matching voice —af_heartspeaks US English whatever the language is set to.speed: Optional speech rate multiplier (Kokoro accepts 0.5 to 2.0; 1.0 is its default). Pauses scale with it too, so a faster rate also shortens the silences at commas.
- pipecat.evals.services.cartesia_service(voice_cfg: dict) TTSService[source]
Build a Cartesia TTS service from the
user_audioconfig.Deprecated since version 1.9.0: Use
user.speech.factoryinstead: a factory that constructsCartesiaHttpTTSService. Will be removed in 2.0.0.- Parameters:
voice_cfg –
The
user.speechconfig mapping:voice: Cartesia voice id.model: Optional model (defaults tosonic-2).api_key: Optional key (falls back to$CARTESIA_API_KEY).language: Optional language code (e.g.zh) orLanguage. When omitted, Cartesia keeps its own default (English).
- Raises:
RuntimeError – If no API key is given in the config or the environment.
- pipecat.evals.services.whisper_service(config: dict) STTService[source]
Build a local Whisper STT service from the
bot_audioconfig.It runs on the CPU by default, leaving the GPU to the judge and the audio models; transcription is off the hot path, so the latency is fine. Set
device: cuda(andcompute_type) in thetranscriptionconfig for GPU.Whisper’s non-speech filter is disabled: it only ever hears the bot’s own speech, and the default threshold drops correct transcriptions of TTS speech run to run.
- Parameters:
config –
The
judge.transcriptionconfig mapping:device:cpu(default) orcuda.compute_type: Whisper compute type (int8on CPU).model: Optional Whisper model (left unset to use Whisper’s own). Whisper’s default is English-only, as is every.enmodel, so a non-Englishlanguageneeds a multilingual model here (e.g.large-v3-turbo).language: Optional language code (e.g.es) orLanguage. When omitted, Whisper keeps its own default (English) — it does not auto-detect, so a non-English bot needs this set.
- Raises:
ValueError – If
languagenames a language the chosen model can’t transcribe (raised byWhisperSTTService).
- pipecat.evals.services.moonshine_service(config: dict) STTService[source]
Build a local Moonshine STT service from the
bot_audioconfig.Small and fast on the CPU, and steadier than Whisper on the short bot-answer segments the harness transcribes.
- Parameters:
config –
The
judge.transcriptionconfig mapping:model: Optional architecture, as aModelor the equivalent string (defaultModel.SMALL_STREAMING). Onlybasehas non-English models, so a non-Englishlanguageneeds it, and raises at construction naming the models it does have if the pairing has none.language: Optional language code (e.g.es) orLanguage. When omitted, Moonshine keeps its own default (English).
Prefer
whisper_service()for a non-English bot: Moonshine’s non-English models transcribe synthesized speech unreliably, returning an empty transcript or dropping the tail of an utterance, and an empty transcript is indistinguishable from a bot that said nothing.
- pipecat.evals.services.classifier_from_config(config: dict | None, *, where: str) BaseClassifier[source]
Build the classifier a
judge.eval:block names.A
factory(a dotted path to a callable taking the config) builds it, and may return aBaseClassifieror an LLM service, which anLLMClassifierthen asks. Otherwise the block names an LLM, asllm_service_from_config()reads it, and anLLMClassifierasks that.- Parameters:
config – Mapping with a
factory, or the keysllm_service_from_config()reads.Noneuses all defaults.where – The config block’s name in the file, for error messages.
- Returns:
The configured classifier.
- Raises:
ValueError – If
serviceis unknown,factoryis not a dotted path, or the factory returned neither a classifier nor an LLM service.
- pipecat.evals.services.llm_service_from_config(config: dict | None, *, where: str) LLMService[Any][source]
Build an LLM service from a
service:block, the judge’s or a persona’s.A
factory(a dotted path to a callable taking the config) builds it; otherwise theservicename picks a provider,ollama, the default.- Parameters:
config –
Mapping with keys
service(default"ollama"),model, optionalendpoint, and an optionalextramapping forwarded to the model as top-level request parameters.Noneuses all defaults.service: openaistill builds an OpenAI service, with a warning.Deprecated since version 1.9.0: Use
factoryinstead ofservice: openai. Will be removed in 2.0.0.where – The config block’s name in the file, for error messages (
"judge.eval","simulator").
- Returns:
The configured LLM service.
- Raises:
ValueError – If
serviceis unknown orfactoryis not a dotted path.
- pipecat.evals.services.ollama_service(config: dict) LLMService[Any][source]
Build a local Ollama LLM service from the
judge:config.An
extra:mapping goes to the model as request parameters, which is howreasoning_effort: nonereaches a thinking model. The default model comes with the extras it needs; a named model gets only the extras asked for.
- pipecat.evals.services.openai_service(config: dict) LLMService[Any][source]
Build an OpenAI LLM service from the
judge:config.Deprecated since version 1.9.0: Use
judge.eval.factory(orsimulator.factory) instead: a factory that constructsOpenAILLMService. Will be removed in 2.0.0.An
extra:mapping is forwarded verbatim as top-level request parameters.