simulation_session

The simulation session: runs a simulated scenario against a bot.

Example:

for scenario in EvalScenarioFile.load("scenarios/simulated/curious_caller.yaml"):
    run = await EvalSimulationSession.from_scenario(scenario, "ws://localhost:7860").run()
    print(f"{'succeeded' if run.succeeded else 'failed'}: {run.reason}")
class pipecat.evals.simulation_session.EvalSimulationSession(scenario: EvalSimulationScenario, bot_url: str, *, params: EvalSessionParams | None = None, persona_llm: LLMService, judge: EvalJudge | None, user_tts: CachingTTSService | None = None, bot_stt: STTService | None = None)[source]

Bases: EvalSession[EvalSimulationResult]

Runs one EvalSimulationScenario against a bot.

The persona LLM answers the bot on its own inside the client’s pipeline, and the judge decides the goal and the criteria at the end. Build one with from_scenario(), which constructs the persona LLM, the judge, and in audio mode the user TTS and the STT, then await run().

__init__(scenario: EvalSimulationScenario, bot_url: str, *, params: EvalSessionParams | None = None, persona_llm: LLMService, judge: EvalJudge | None, user_tts: CachingTTSService | None = None, bot_stt: STTService | None = None)[source]

Initialize the simulation session.

Parameters:

  • scenario – The parsed simulation to run.

  • bot_url – WebSocket URL of the bot’s eval transport.

  • params – How the run behaves; None for the defaults.

  • persona_llm – The persona LLM service, run inside the eval pipeline.

  • judge – The judge for the goal and the quality criteria, or None (the run then reports no verdict).

  • user_tts – The user-audio TTS, or None for text mode.

  • bot_stt – The bot-audio STT, or None in text mode.

classmethod from_scenario(scenario: EvalSimulationScenario, bot_url: str, *, params: EvalSessionParams | None = None, persona_llm: LLMService | None = None, judge: EvalJudge | None = None, user_tts: CachingTTSService | None = None, bot_stt: STTService | None = None) → EvalSimulationSession[source]

Build a ready-to-run session from a scenario, constructing the services it needs.

Pass persona_llm, judge, user_tts, or bot_stt to use your own.

Parameters:

  • scenario – The parsed simulation to run.

  • bot_url – WebSocket URL of the bot’s eval transport.

  • params – How the run behaves; None for the defaults.

  • persona_llm – Override the persona LLM (default: built from simulation.simulator).

  • judge – Override the judge (default: built from simulation.judge).

  • user_tts – Override the user-audio TTS (default: built from simulation.user_speech in audio mode).

  • bot_stt – Override the bot-audio STT (default: built from simulation.transcriber in audio mode).

Returns:

A configured session, ready for run().