serializer

Frame serializer that bridges the eval harness to a bot over RTVI.

The eval harness (pipecat.evals.harness) talks to a bot using the RTVI protocol over a plain WebSocket (SingleClientWebsocketServerTransport). This serializer is the only glue needed:

• Inbound (harness → bot): JSON RTVI messages are wrapped in an InputTransportMessageFrame so the bot’s RTVIProcessor parses and routes them (send-text, raw-audio, dtmf, client-ready, …). Two control messages are the exception: a client-message with t = "eval-context" is short-circuited into an LLMMessagesUpdateFrame (reseeding the bot’s context), and one with t = "eval-configure" into an RTVIConfigureObserverFrame (raising the function-call report level for the eval). Both keep eval-specific behavior out of the bot, and the latter is the trust boundary that lets bots keep the secure default report level in production.

• Outbound (bot → harness): RTVI server messages (carried as OutputTransportMessage*Frame with the rtvi-ai label) are emitted as their raw JSON. Everything else — notably bot audio — is dropped, since the harness only asserts on semantic events.

This lives under pipecat.evals rather than pipecat.serializers because it carries eval-specific behavior (the context short-circuit, dropping audio) and is not a general-purpose RTVI transport serializer.

class pipecat.evals.serializer.RTVIEvalSerializer(**kwargs)

Bases: FrameSerializer

Bridges JSON RTVI messages and pipeline frames for the eval harness.

Use as the serializer of a SingleClientWebsocketServerTransport when running a bot under the eval harness. The bot pipeline must include an RTVIProcessor and pass an RTVIObserver to the task.

__init__(**kwargs)

Initialize the serializer.

Parameters:

**kwargs – Additional arguments passed to FrameSerializer.

set_capture_audio(capture: bool) → None

Enable/disable forwarding the bot’s synthesized audio to the harness.

get_user_image() → tuple[bytes, str] | None

The image registered for the current turn as (bytes, mime), or None.

async serialize(frame: Frame) → str | bytes | None

Serialize an outbound frame to JSON for the harness.

Only RTVI server messages are forwarded; all other frames (audio, control) are dropped.

Parameters:

frame – The frame to serialize.

Returns:

JSON text for an RTVI server message, or None to drop the frame.

async deserialize(data: str | bytes) → Frame | None

Deserialize an inbound JSON RTVI message into a frame.

Parameters:

data – JSON text (or bytes) sent by the harness.

Returns:

An LLMMessagesUpdateFrame for the eval-context control message, an InputTransportMessageFrame wrapping any other RTVI message, or None if the payload is not a valid RTVI message.