models

RTVI protocol v1 message models.

Contains all RTVI protocol v1 message definitions and data structures. Import this module under the RTVI alias to use as a namespace:

import pipecat.processors.frameworks.rtvi.models as RTVI

msg = RTVI.BotReady(id="1", data=RTVI.BotReadyData(version=RTVI.PROTOCOL_VERSION))
class pipecat.processors.frameworks.rtvi.models.Message(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: str, id: str, data: dict[str, Any] | None = None)[source]

Bases: BaseModel

Base RTVI message structure.

Represents the standard format for RTVI protocol messages.

label: Literal['rtvi-ai']
type: str
id: str
data: dict[str, Any] | None
class pipecat.processors.frameworks.rtvi.models.RawClientMessageData(*, t: str, d: Any | None = None)[source]

Bases: BaseModel

Data structure expected from client messages sent to the RTVI server.

t: str
d: Any | None
class pipecat.processors.frameworks.rtvi.models.ClientMessage(*, msg_id: str, type: str, data: Any | None = None)[source]

Bases: BaseModel

Cleansed data structure for client messages for handling.

msg_id: str
type: str
data: Any | None
class pipecat.processors.frameworks.rtvi.models.RawServerResponseData(*, t: str, d: Any | None = None)[source]

Bases: BaseModel

Data structure for server responses to client messages.

t: str
d: Any | None
class pipecat.processors.frameworks.rtvi.models.ServerResponse(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['server-response'] = 'server-response', id: str, data: RawServerResponseData)[source]

Bases: BaseModel

The RTVI-formatted message response from the server to the client.

This message is used to respond to custom messages sent by the client.

label: Literal['rtvi-ai']
type: Literal['server-response']
id: str
data: RawServerResponseData
class pipecat.processors.frameworks.rtvi.models.AboutClientData(*, library: str, library_version: str | None = None, platform: str | None = None, platform_version: str | None = None, platform_details: Any | None = None)[source]

Bases: BaseModel

Data about the RTVI client.

Contains information about the client, including which RTVI library it is using, what platform it is on and any additional details, if available.

library: str
library_version: str | None
platform: str | None
platform_version: str | None
platform_details: Any | None
class pipecat.processors.frameworks.rtvi.models.ClientReadyData(*, version: str, about: AboutClientData)[source]

Bases: BaseModel

Data format of client ready messages.

Contains the RTVI protocol version and client information.

version: str
about: AboutClientData
class pipecat.processors.frameworks.rtvi.models.ErrorResponseData(*, error: str)[source]

Bases: BaseModel

Data for an RTVI error response.

Contains the error message to send back to the client.

error: str
class pipecat.processors.frameworks.rtvi.models.ErrorResponse(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['error-response'] = 'error-response', id: str, data: ErrorResponseData)[source]

Bases: BaseModel

RTVI error response message.

RTVI formatted error response message for relaying failed client requests.

label: Literal['rtvi-ai']
type: Literal['error-response']
id: str
data: ErrorResponseData
class pipecat.processors.frameworks.rtvi.models.ErrorData(*, error: str, fatal: bool)[source]

Bases: BaseModel

Data for an RTVI error event.

Contains error information including whether it’s fatal.

error: str
fatal: bool
class pipecat.processors.frameworks.rtvi.models.Error(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['error'] = 'error', data: ErrorData)[source]

Bases: BaseModel

RTVI error event message.

RTVI formatted error message for relaying errors in the pipeline.

label: Literal['rtvi-ai']
type: Literal['error']
data: ErrorData
class pipecat.processors.frameworks.rtvi.models.BotReadyData(*, version: str, about: Mapping[str, Any] | None = None)[source]

Bases: BaseModel

Data for bot ready notification.

Contains protocol version and initial configuration.

version: str
about: Mapping[str, Any] | None
class pipecat.processors.frameworks.rtvi.models.BotReady(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['bot-ready'] = 'bot-ready', id: str, data: BotReadyData)[source]

Bases: BaseModel

Message indicating bot is ready for interaction.

Sent after bot initialization is complete.

label: Literal['rtvi-ai']
type: Literal['bot-ready']
id: str
data: BotReadyData
class pipecat.processors.frameworks.rtvi.models.LLMFunctionCallMessageData(*, function_name: str, tool_call_id: str, args: Mapping[str, Any])[source]

Bases: BaseModel

Data for LLM function call notification.

Contains function call details including name, ID, and arguments.

Deprecated since version 0.0.102: Use LLMFunctionCallInProgressMessageData instead.

function_name: str
tool_call_id: str
args: Mapping[str, Any]
class pipecat.processors.frameworks.rtvi.models.LLMFunctionCallMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['llm-function-call'] = 'llm-function-call', data: LLMFunctionCallMessageData)[source]

Bases: BaseModel

Message notifying of an LLM function call.

Sent when the LLM makes a function call.

Deprecated since version 0.0.102: Use LLMFunctionCallInProgressMessage with the llm-function-call-in-progress event type instead.

label: Literal['rtvi-ai']
type: Literal['llm-function-call']
data: LLMFunctionCallMessageData
class pipecat.processors.frameworks.rtvi.models.SendTextOptions(*, run_immediately: bool = True, audio_response: bool = True)[source]

Bases: BaseModel

Options for sending text input to the LLM.

Contains options for how the pipeline should process the text input.

run_immediately: bool
audio_response: bool
class pipecat.processors.frameworks.rtvi.models.SendTextData(*, content: str, options: SendTextOptions | None = None)[source]

Bases: BaseModel

Data format for sending text input to the LLM.

Contains the text content to send and any options for how the pipeline should process it.

content: str
options: SendTextOptions | None
class pipecat.processors.frameworks.rtvi.models.LLMFunctionCallStartMessageData(*, function_name: str | None = None)[source]

Bases: BaseModel

Data for LLM function call start notification.

Contains the function name being called. Fields may be omitted based on the configured function_call_report_level for security.

function_name: str | None
class pipecat.processors.frameworks.rtvi.models.LLMFunctionCallStartMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['llm-function-call-started'] = 'llm-function-call-started', data: LLMFunctionCallStartMessageData)[source]

Bases: BaseModel

Message notifying that an LLM function call has started.

Sent when the LLM begins a function call.

label: Literal['rtvi-ai']
type: Literal['llm-function-call-started']
data: LLMFunctionCallStartMessageData
class pipecat.processors.frameworks.rtvi.models.LLMFunctionCallResultData(*, function_name: str, tool_call_id: str, arguments: dict, result: dict | str)[source]

Bases: BaseModel

Data for LLM function call result.

Contains function call details and result.

function_name: str
tool_call_id: str
arguments: dict
result: dict | str
class pipecat.processors.frameworks.rtvi.models.LLMFunctionCallInProgressMessageData(*, tool_call_id: str, function_name: str | None = None, arguments: Mapping[str, Any] | None = None)[source]

Bases: BaseModel

Data for LLM function call in-progress notification.

Contains function call details including name, ID, and arguments. Fields may be omitted based on the configured function_call_report_level for security.

tool_call_id: str
function_name: str | None
arguments: Mapping[str, Any] | None
class pipecat.processors.frameworks.rtvi.models.LLMFunctionCallInProgressMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['llm-function-call-in-progress'] = 'llm-function-call-in-progress', data: LLMFunctionCallInProgressMessageData)[source]

Bases: BaseModel

Message notifying that an LLM function call is in progress.

Sent when the LLM function call execution begins.

label: Literal['rtvi-ai']
type: Literal['llm-function-call-in-progress']
data: LLMFunctionCallInProgressMessageData
class pipecat.processors.frameworks.rtvi.models.LLMFunctionCallStoppedMessageData(*, tool_call_id: str, cancelled: bool, function_name: str | None = None, result: Any | None = None)[source]

Bases: BaseModel

Data for LLM function call stopped notification.

Contains details about the function call that stopped, including whether it was cancelled or completed with a result. Fields may be omitted based on the configured function_call_report_level for security.

tool_call_id: str
cancelled: bool
function_name: str | None
result: Any | None
class pipecat.processors.frameworks.rtvi.models.LLMFunctionCallStoppedMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['llm-function-call-stopped'] = 'llm-function-call-stopped', data: LLMFunctionCallStoppedMessageData)[source]

Bases: BaseModel

Message notifying that an LLM function call has stopped.

Sent when a function call completes (with result) or is cancelled.

label: Literal['rtvi-ai']
type: Literal['llm-function-call-stopped']
data: LLMFunctionCallStoppedMessageData
class pipecat.processors.frameworks.rtvi.models.BotLLMStartedMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['bot-llm-started'] = 'bot-llm-started')[source]

Bases: BaseModel

Message indicating bot LLM processing has started.

label: Literal['rtvi-ai']
type: Literal['bot-llm-started']
class pipecat.processors.frameworks.rtvi.models.BotLLMStoppedMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['bot-llm-stopped'] = 'bot-llm-stopped')[source]

Bases: BaseModel

Message indicating bot LLM processing has stopped.

label: Literal['rtvi-ai']
type: Literal['bot-llm-stopped']
class pipecat.processors.frameworks.rtvi.models.BotTTSStartedMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['bot-tts-started'] = 'bot-tts-started')[source]

Bases: BaseModel

Message indicating bot TTS processing has started.

label: Literal['rtvi-ai']
type: Literal['bot-tts-started']
class pipecat.processors.frameworks.rtvi.models.BotTTSStoppedMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['bot-tts-stopped'] = 'bot-tts-stopped')[source]

Bases: BaseModel

Message indicating bot TTS processing has stopped.

label: Literal['rtvi-ai']
type: Literal['bot-tts-stopped']
class pipecat.processors.frameworks.rtvi.models.TextMessageData(*, text: str)[source]

Bases: BaseModel

Data for text-based RTVI messages.

Contains text content.

text: str
class pipecat.processors.frameworks.rtvi.models.BotOutputMessageData(*, text: str, spoken: bool = False, aggregated_by: AggregationType | str)[source]

Bases: TextMessageData

Data for bot output RTVI messages.

Extends TextMessageData to include metadata about the output.

spoken: bool
aggregated_by: AggregationType | str
class pipecat.processors.frameworks.rtvi.models.BotOutputMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['bot-output'] = 'bot-output', data: BotOutputMessageData)[source]

Bases: BaseModel

Message containing bot output text.

An event meant to holistically represent what the bot is outputting, along with metadata about the output and if it has been spoken.

label: Literal['rtvi-ai']
type: Literal['bot-output']
data: BotOutputMessageData
class pipecat.processors.frameworks.rtvi.models.BotTranscriptionMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['bot-transcription'] = 'bot-transcription', data: TextMessageData)[source]

Bases: BaseModel

Message containing bot transcription text.

Sent when the bot’s speech is transcribed.

label: Literal['rtvi-ai']
type: Literal['bot-transcription']
data: TextMessageData
class pipecat.processors.frameworks.rtvi.models.BotLLMTextMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['bot-llm-text'] = 'bot-llm-text', data: TextMessageData)[source]

Bases: BaseModel

Message containing bot LLM text output.

Sent when the bot’s LLM generates text.

label: Literal['rtvi-ai']
type: Literal['bot-llm-text']
data: TextMessageData
class pipecat.processors.frameworks.rtvi.models.BotTTSTextMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['bot-tts-text'] = 'bot-tts-text', data: TextMessageData)[source]

Bases: BaseModel

Message containing bot TTS text output.

Sent when text is being processed by TTS.

label: Literal['rtvi-ai']
type: Literal['bot-tts-text']
data: TextMessageData
class pipecat.processors.frameworks.rtvi.models.AudioMessageData(*, audio: str, sample_rate: int, num_channels: int)[source]

Bases: BaseModel

Data for audio-based RTVI messages.

Contains audio data and metadata.

audio: str
sample_rate: int
num_channels: int
class pipecat.processors.frameworks.rtvi.models.BotTTSAudioMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['bot-tts-audio'] = 'bot-tts-audio', data: AudioMessageData)[source]

Bases: BaseModel

Message containing bot TTS audio output.

Sent when the bot’s TTS generates audio.

label: Literal['rtvi-ai']
type: Literal['bot-tts-audio']
data: AudioMessageData
class pipecat.processors.frameworks.rtvi.models.UserTranscriptionMessageData(*, text: str, user_id: str, timestamp: str, final: bool)[source]

Bases: BaseModel

Data for user transcription messages.

Contains transcription text and metadata.

text: str
user_id: str
timestamp: str
final: bool
class pipecat.processors.frameworks.rtvi.models.UserTranscriptionMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['user-transcription'] = 'user-transcription', data: UserTranscriptionMessageData)[source]

Bases: BaseModel

Message containing user transcription.

Sent when user speech is transcribed.

label: Literal['rtvi-ai']
type: Literal['user-transcription']
data: UserTranscriptionMessageData
class pipecat.processors.frameworks.rtvi.models.UserLLMTextMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['user-llm-text'] = 'user-llm-text', data: TextMessageData)[source]

Bases: BaseModel

Message containing user text input for LLM.

Sent when user text is processed by the LLM.

label: Literal['rtvi-ai']
type: Literal['user-llm-text']
data: TextMessageData
class pipecat.processors.frameworks.rtvi.models.UserStartedSpeakingMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['user-started-speaking'] = 'user-started-speaking')[source]

Bases: BaseModel

Message indicating user has started speaking.

label: Literal['rtvi-ai']
type: Literal['user-started-speaking']
class pipecat.processors.frameworks.rtvi.models.UserStoppedSpeakingMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['user-stopped-speaking'] = 'user-stopped-speaking')[source]

Bases: BaseModel

Message indicating user has stopped speaking.

label: Literal['rtvi-ai']
type: Literal['user-stopped-speaking']
class pipecat.processors.frameworks.rtvi.models.UserMuteStartedMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['user-mute-started'] = 'user-mute-started')[source]

Bases: BaseModel

Message indicating user has been muted.

label: Literal['rtvi-ai']
type: Literal['user-mute-started']
class pipecat.processors.frameworks.rtvi.models.UserMuteStoppedMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['user-mute-stopped'] = 'user-mute-stopped')[source]

Bases: BaseModel

Message indicating user has been unmuted.

label: Literal['rtvi-ai']
type: Literal['user-mute-stopped']
class pipecat.processors.frameworks.rtvi.models.BotStartedSpeakingMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['bot-started-speaking'] = 'bot-started-speaking')[source]

Bases: BaseModel

Message indicating bot has started speaking.

label: Literal['rtvi-ai']
type: Literal['bot-started-speaking']
class pipecat.processors.frameworks.rtvi.models.BotStoppedSpeakingMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['bot-stopped-speaking'] = 'bot-stopped-speaking')[source]

Bases: BaseModel

Message indicating bot has stopped speaking.

label: Literal['rtvi-ai']
type: Literal['bot-stopped-speaking']
class pipecat.processors.frameworks.rtvi.models.MetricsMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['metrics'] = 'metrics', data: Mapping[str, Any])[source]

Bases: BaseModel

Message containing performance metrics.

Sent to provide performance and usage metrics.

label: Literal['rtvi-ai']
type: Literal['metrics']
data: Mapping[str, Any]
class pipecat.processors.frameworks.rtvi.models.ServerMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['server-message'] = 'server-message', data: Any)[source]

Bases: BaseModel

Generic server message.

Used for custom server-to-client messages.

label: Literal['rtvi-ai']
type: Literal['server-message']
data: Any
class pipecat.processors.frameworks.rtvi.models.AudioLevelMessageData(*, value: float)[source]

Bases: BaseModel

Data format for sending audio levels.

value: float
class pipecat.processors.frameworks.rtvi.models.UserAudioLevelMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['user-audio-level'] = 'user-audio-level', data: AudioLevelMessageData)[source]

Bases: BaseModel

Message indicating user audio level.

label: Literal['rtvi-ai']
type: Literal['user-audio-level']
data: AudioLevelMessageData
class pipecat.processors.frameworks.rtvi.models.BotAudioLevelMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['bot-audio-level'] = 'bot-audio-level', data: AudioLevelMessageData)[source]

Bases: BaseModel

Message indicating bot audio level.

label: Literal['rtvi-ai']
type: Literal['bot-audio-level']
data: AudioLevelMessageData
class pipecat.processors.frameworks.rtvi.models.SystemLogMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['system-log'] = 'system-log', data: TextMessageData)[source]

Bases: BaseModel

Message including a system log.

label: Literal['rtvi-ai']
type: Literal['system-log']
data: TextMessageData
class pipecat.processors.frameworks.rtvi.models.UIEventData(*, event: str, payload: Any | None = None)[source]

Bases: BaseModel

Inner data for a ui-event message.

Parameters:

  • event – App-defined event.

  • payload – App-defined payload, schemaless by design.

event: str
payload: Any | None
class pipecat.processors.frameworks.rtvi.models.UICommandData(*, command: str, payload: Any | None = None)[source]

Bases: BaseModel

Inner data for a ui-command message.

Parameters:

  • command – App-defined command.

  • payload – App-defined payload (already a plain dict by the time it lands on the wire). The standard command payload models below produce the right shape via model_dump().

command: str
payload: Any | None
class pipecat.processors.frameworks.rtvi.models.A11yNode(*, ref: str, role: str, name: str | None = None, value: str | None = None, state: list[str] | None = None, level: int | None = None, colcount: int | None = None, rowcount: int | None = None, children: list[A11yNode] | None = None, **extra_data: Any)[source]

Bases: BaseModel

One node in the UI accessibility snapshot tree.

Mirrors the client-side A11yNode wire shape. Extra fields are allowed so clients can add platform-specific or future metadata without breaking older servers.

Parameters:

  • ref – Stable client-assigned element reference.

  • role – ARIA-style role for the node.

  • name – Optional accessible name.

  • value – Optional current value for inputs/progress/etc.

  • state – Optional short state tags (e.g. "focused", "disabled", "offscreen").

  • level – Optional heading level.

  • colcount – Optional column count for grid-like containers.

  • rowcount – Optional row count for grid-like containers.

  • children – Optional child nodes.

ref: str
role: str
name: str | None
value: str | None
state: list[str] | None
level: int | None
colcount: int | None
rowcount: int | None
children: list[A11yNode] | None
class pipecat.processors.frameworks.rtvi.models.A11ySelection(*, ref: str, text: str, start_offset: int | None = None, end_offset: int | None = None, **extra_data: Any)[source]

Bases: BaseModel

The user’s current text selection in the UI snapshot.

Extra fields are allowed for forward compatibility with client snapshot additions.

Parameters:

  • ref – Ref of the element that carries the selection.

  • text – Selected text.

  • start_offset – Optional selection start offset.

  • end_offset – Optional selection end offset.

ref: str
text: str
start_offset: int | None
end_offset: int | None
class pipecat.processors.frameworks.rtvi.models.A11ySnapshot(*, root: A11yNode, captured_at: int, selection: A11ySelection | None = None, **extra_data: Any)[source]

Bases: BaseModel

Client accessibility snapshot sent in a ui-snapshot message.

Mirrors the client-side A11ySnapshot wire shape. Extra fields are allowed so clients can add compatible metadata over time.

Parameters:

  • root – Root accessibility node.

  • captured_at – Client-side epoch milliseconds when captured.

  • selection – Optional current text selection.

root: A11yNode
captured_at: int
selection: A11ySelection | None
class pipecat.processors.frameworks.rtvi.models.UISnapshotData(*, tree: A11ySnapshot)[source]

Bases: BaseModel

Inner data for a ui-snapshot message.

The accessibility snapshot tree mirrors the client-side A11ySnapshot wire shape and is kept forward-compatible by allowing extra fields on the snapshot models.

Parameters:

tree – The serialized accessibility tree.

tree: A11ySnapshot
class pipecat.processors.frameworks.rtvi.models.UICancelJobGroupData(*, job_id: str, reason: str | None = None)[source]

Bases: BaseModel

Inner data for a ui-cancel-job-group message.

Parameters:

  • job_id – The job group id the client wants cancelled.

  • reason – Optional human-readable reason.

job_id: str
reason: str | None
class pipecat.processors.frameworks.rtvi.models.UIJobGroupStartedData(*, kind: Literal['group_started'] = 'group_started', job_id: str, workers: list[str] | None = None, label: str | None = None, cancellable: bool = True, at: int = 0)[source]

Bases: BaseModel

data for a ui-job-group envelope with kind group_started.

Parameters:

  • kind – Always "group_started".

  • job_id – Shared job-group identifier for the group.

  • workers – Names of the workers the work was dispatched to.

  • label – Optional human-readable label for the group.

  • cancellable – Whether the client may request cancellation.

  • at – Epoch milliseconds when the group started.

kind: Literal['group_started']
job_id: str
workers: list[str] | None
label: str | None
cancellable: bool
at: int
class pipecat.processors.frameworks.rtvi.models.UIJobUpdateData(*, kind: Literal['job_update'] = 'job_update', job_id: str, worker_name: str, data: Any | None = None, at: int = 0)[source]

Bases: BaseModel

data for a ui-job-group envelope with kind job_update.

Parameters:

  • kind – Always "job_update".

  • job_id – The shared job-group identifier.

  • worker_name – The worker that produced the update.

  • data – The worker’s update payload, forwarded verbatim.

  • at – Epoch milliseconds when the update was emitted.

kind: Literal['job_update']
job_id: str
worker_name: str
data: Any | None
at: int
class pipecat.processors.frameworks.rtvi.models.UIJobCompletedData(*, kind: Literal['job_completed'] = 'job_completed', job_id: str, worker_name: str, status: str, response: Any | None = None, at: int = 0)[source]

Bases: BaseModel

data for a ui-job-group envelope with kind job_completed.

Parameters:

  • kind – Always "job_completed".

  • job_id – The shared job-group identifier.

  • worker_name – The worker that produced the response.

  • status – Completion status string.

  • response – The worker’s response payload.

  • at – Epoch milliseconds when the response was received.

kind: Literal['job_completed']
job_id: str
worker_name: str
status: str
response: Any | None
at: int
class pipecat.processors.frameworks.rtvi.models.UIJobGroupCompletedData(*, kind: Literal['group_completed'] = 'group_completed', job_id: str, at: int = 0)[source]

Bases: BaseModel

data for a ui-job-group envelope with kind group_completed.

Parameters:

  • kind – Always "group_completed".

  • job_id – The shared job-group identifier.

  • at – Epoch milliseconds when the group completed.

kind: Literal['group_completed']
job_id: str
at: int
pipecat.processors.frameworks.rtvi.models.UIJobGroupData = pipecat.processors.frameworks.rtvi.models.UIJobGroupStartedData | pipecat.processors.frameworks.rtvi.models.UIJobUpdateData | pipecat.processors.frameworks.rtvi.models.UIJobCompletedData | pipecat.processors.frameworks.rtvi.models.UIJobGroupCompletedData

Discriminated union over the four job-group lifecycle data shapes, keyed by the kind field.

class pipecat.processors.frameworks.rtvi.models.UIEventMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['ui-event'] = 'ui-event', id: str, data: UIEventData)[source]

Bases: BaseModel

RTVI ui-event message (client → server).

label: Literal['rtvi-ai']
type: Literal['ui-event']
id: str
data: UIEventData
class pipecat.processors.frameworks.rtvi.models.UICommandMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['ui-command'] = 'ui-command', data: UICommandData)[source]

Bases: BaseModel

RTVI ui-command message (server → client).

label: Literal['rtvi-ai']
type: Literal['ui-command']
data: UICommandData
class pipecat.processors.frameworks.rtvi.models.UISnapshotMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['ui-snapshot'] = 'ui-snapshot', id: str, data: UISnapshotData)[source]

Bases: BaseModel

RTVI ui-snapshot message (client → server).

label: Literal['rtvi-ai']
type: Literal['ui-snapshot']
id: str
data: UISnapshotData
class pipecat.processors.frameworks.rtvi.models.UICancelJobGroupMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['ui-cancel-job-group'] = 'ui-cancel-job-group', id: str, data: UICancelJobGroupData)[source]

Bases: BaseModel

RTVI ui-cancel-job-group message (client → server).

label: Literal['rtvi-ai']
type: Literal['ui-cancel-job-group']
id: str
data: UICancelJobGroupData
class pipecat.processors.frameworks.rtvi.models.UIJobGroupMessage(*, label: Literal['rtvi-ai'] = 'rtvi-ai', type: Literal['ui-job-group'] = 'ui-job-group', data: UIJobGroupStartedData | UIJobUpdateData | UIJobCompletedData | UIJobGroupCompletedData)[source]

Bases: BaseModel

RTVI ui-job-group message (server → client).

The data field is one of the four job-group lifecycle discriminated by the kind field.

label: Literal['rtvi-ai']
type: Literal['ui-job-group']
data: UIJobGroupStartedData | UIJobUpdateData | UIJobCompletedData | UIJobGroupCompletedData
class pipecat.processors.frameworks.rtvi.models.Toast(*, title: str, subtitle: str | None = None, description: str | None = None, image_url: str | None = None, duration_ms: int | None = None)[source]

Bases: BaseModel

A transient notification surface shown on the client.

Parameters:

  • title – Required headline.

  • subtitle – Optional second line beneath the title.

  • description – Optional body text.

  • image_url – Optional leading image.

  • duration_ms – Optional dismiss timer. Client default applies when None.

title: str
subtitle: str | None
description: str | None
image_url: str | None
duration_ms: int | None
class pipecat.processors.frameworks.rtvi.models.Navigate(*, view: str, params: dict | None = None)[source]

Bases: BaseModel

Client-side navigation to a named view.

Parameters:

  • view – App-defined view name (route, screen id, tab key, etc.).

  • params – Optional view-specific parameters.

view: str
params: dict | None
class pipecat.processors.frameworks.rtvi.models.ScrollTo(*, ref: str | None = None, target_id: str | None = None, behavior: str | None = None)[source]

Bases: BaseModel

Scroll a target element into view.

The client resolves the target by ref first (a snapshot ref like "e42" assigned by the a11y walker), then falls back to target_id (document.getElementById). Supply whichever you have; ref is the normal choice when acting on a node from <ui_state>.

Parameters:

  • ref – Snapshot ref from <ui_state>.

  • target_id – Element id registered on the client.

  • behavior – Optional scroll behavior hint. Typical values: "smooth" or "instant". Clients may ignore.

ref: str | None
target_id: str | None
behavior: str | None
class pipecat.processors.frameworks.rtvi.models.Highlight(*, ref: str | None = None, target_id: str | None = None, duration_ms: int | None = None)[source]

Bases: BaseModel

Briefly emphasize a target element (flash, glow, pulse).

Parameters:

  • ref – Snapshot ref from <ui_state>.

  • target_id – Element id registered on the client.

  • duration_ms – Optional highlight duration. Client default applies when None.

ref: str | None
target_id: str | None
duration_ms: int | None
class pipecat.processors.frameworks.rtvi.models.Focus(*, ref: str | None = None, target_id: str | None = None)[source]

Bases: BaseModel

Move input focus to a target element.

Parameters:

  • ref – Snapshot ref from <ui_state>.

  • target_id – Element id registered on the client.

ref: str | None
target_id: str | None
class pipecat.processors.frameworks.rtvi.models.Click(*, ref: str | None = None, target_id: str | None = None)[source]

Bases: BaseModel

Click an element on the client.

Closes the form-fill loop for non-text inputs (checkboxes, radios) and exposes the rest of the action vocabulary (submit buttons, links, app-specific clickable nodes). The standard handler silently no-ops on disabled targets so the worker can’t bypass UI affordances the user is meant to control.

For native <select>, prefer SetInputValue (clicking options doesn’t reliably change the selection); for custom comboboxes (ARIA listbox + popup), apps wire their own command matching the library’s interaction model.

Parameters:

  • ref – Snapshot ref from <ui_state>.

  • target_id – Element id registered on the client. Used as a fallback when ref is not set or has gone stale.

ref: str | None
target_id: str | None
class pipecat.processors.frameworks.rtvi.models.SetInputValue(*, value: str = '', ref: str | None = None, target_id: str | None = None, replace: bool = True)[source]

Bases: BaseModel

Write a value into a text input or textarea on the client.

Use this for form-filling: the worker has decided what should go into a field (clarifying answer, tax form entry, etc.) and asks the client to populate it. With replace=True (the default), the existing value is overwritten; with replace=False the value is appended.

The standard handler silently no-ops on disabled, readonly, and <input type="hidden"> targets so the worker can’t write into fields the user can’t.

Parameters:

  • value – The text to write.

  • ref – Snapshot ref from <ui_state>. Typically the ref of an <input> or <textarea>.

  • target_id – Element id registered on the client. Used as a fallback when ref is not set or has gone stale.

  • replace – When True (the default), overwrite the current value. When False, append to it.

value: str
ref: str | None
target_id: str | None
replace: bool
class pipecat.processors.frameworks.rtvi.models.SelectText(*, ref: str | None = None, target_id: str | None = None, start_offset: int | None = None, end_offset: int | None = None)[source]

Bases: BaseModel

Select text on the page so the user can see what the worker means.

Mirror of the selection field surfaced in the snapshot. Use this to point the user’s attention at a specific paragraph or range after the worker has decided what it’s referring to.

With start_offset and end_offset omitted, the entire target’s text content is selected (Range.selectNodeContents for document elements; el.select() for <input> / <textarea>).

Parameters:

  • ref – Snapshot ref from <ui_state>. Typically the ref of a paragraph or input element.

  • target_id – Element id registered on the client. Used as a fallback when ref is not set or has gone stale.

  • start_offset – Character offset within the target’s text where the selection should start. For <input> and <textarea> this is the value offset; for document elements it is computed against the concatenation of descendant text nodes in document order.

  • end_offset – End character offset, exclusive. Same coordinate system as start_offset.

ref: str | None
target_id: str | None
start_offset: int | None
end_offset: int | None