frames

Core frame definitions for the Pipecat AI framework.

This module contains all frame types used throughout the Pipecat pipeline system, including data frames, system frames, and control frames for audio, video, text, and LLM processing.

pipecat.frames.frames.format_pts(pts: int | None)

Format presentation timestamp (PTS) in nanoseconds to a human-readable string.

Converts a PTS value in nanoseconds to a string representation.

Parameters:

pts – Presentation timestamp in nanoseconds, or None if not set.

class pipecat.frames.frames.Frame

Bases: object

Base frame class for all frames in the Pipecat pipeline.

All frames inherit from this base class and automatically receive unique identifiers, names, and metadata support.

Parameters:

• id – Unique identifier for the frame instance.

• name – Human-readable name combining class name and instance count.

• pts – Presentation timestamp in nanoseconds.

• broadcast_sibling_id – ID of the paired frame when this frame was broadcast in both directions. Set automatically by broadcast_frame() and broadcast_frame_instance().

• metadata – Dictionary for arbitrary frame metadata.

• transport_source – Name of the transport source that created this frame.

• transport_destination – Name of the transport destination for this frame.

class pipecat.frames.frames.SystemFrame

Bases: Frame

System frame class for immediate processing.

A frame that takes higher priority than other frames. System frames are handled in order and are not affected by user interruptions.

class pipecat.frames.frames.DataFrame

Bases: Frame

Data frame class for processing data in order.

A frame that is processed in order and usually contains data such as LLM context, text, audio or images. Data frames are cancelled by user interruptions.

class pipecat.frames.frames.ControlFrame

Bases: Frame

Control frame class for processing control information in order.

A frame that, similar to data frames, is processed in order and usually contains control information such as update settings or to end the pipeline after everything is flushed. Control frames are cancelled by user interruptions.

class pipecat.frames.frames.UninterruptibleFrame

Bases: object

A marker for data or control frames that must not be interrupted.

Frames with this mixin are still ordered normally, but unlike other frames, they are preserved during interruptions: they remain in internal queues and any task processing them will not be cancelled. This ensures the frame is always delivered and processed to completion.

class pipecat.frames.frames.AudioRawFrame(audio: bytes, sample_rate: int, num_channels: int)

Bases: object

A frame containing a chunk of raw audio.

Parameters:

• audio – Raw audio bytes in PCM format.

• sample_rate – Audio sample rate in Hz.

• num_channels – Number of audio channels.

• num_frames – Number of audio frames (calculated automatically).

num_frames: int = 0

class pipecat.frames.frames.ImageRawFrame(image: bytes, size: tuple[int, int], format: str | None)

Bases: object

A frame containing a raw image.

Parameters:

• image – Raw image bytes.

• size – Image dimensions as (width, height) tuple.

• format – Image format (e.g., ‘RGB’, ‘RGBA’).

class pipecat.frames.frames.OutputAudioRawFrame(audio: bytes, sample_rate: int, num_channels: int)

Bases: DataFrame, AudioRawFrame

Audio data frame for output to transport.

A chunk of raw audio that will be played by the output transport. If the transport supports multiple audio destinations (e.g. multiple audio tracks) the destination name can be specified in transport_destination.

class pipecat.frames.frames.OutputImageRawFrame(image: bytes, size: tuple[int, int], format: str | None)

Bases: DataFrame, ImageRawFrame

Image data frame for output to transport.

An image that will be shown by the transport. If the transport supports multiple video destinations (e.g. multiple video tracks) the destination name can be specified in transport_destination.

Parameters:

sync_with_audio – If True, the image is queued with audio frames so it is only displayed after all preceding audio has been sent. Defaults to False (image is displayed immediately when the output transport receives it).

sync_with_audio: bool = False

class pipecat.frames.frames.TTSAudioRawFrame(audio: bytes, sample_rate: int, num_channels: int, context_id: str | None = None)

Bases: OutputAudioRawFrame

Audio data frame generated by Text-to-Speech services.

A chunk of output audio generated by a TTS service, ready for playback.

Parameters:

context_id – Unique identifier for the TTS context that generated this audio.

context_id: str | None = None

class pipecat.frames.frames.SpeechOutputAudioRawFrame(audio: bytes, sample_rate: int, num_channels: int)

Bases: OutputAudioRawFrame

An audio frame part of a speech audio stream.

This frame is part of a continuous stream of audio frames containing speech. The audio stream might also contain silence frames, so a process to distinguish between speech and silence might be needed.

class pipecat.frames.frames.URLImageRawFrame(image: bytes, size: tuple[int, int], format: str | None, url: str | None = None)

Bases: OutputImageRawFrame

Image frame with an associated URL.

An output image with an associated URL. These images are usually generated by third-party services that provide a URL to download the image.

Parameters:

url – URL where the image can be downloaded from.

url: str | None = None

class pipecat.frames.frames.SpriteFrame(images: list[OutputImageRawFrame])

Bases: DataFrame

Animated sprite frame containing multiple images.

An animated sprite that will be shown by the transport if the transport’s camera is enabled. Will play at the framerate specified in the transport’s camera_out_framerate constructor parameter.

Parameters:

images – List of image frames that make up the sprite animation.

class pipecat.frames.frames.TextFrame(text: str)

Bases: DataFrame

Text data frame for passing text through the pipeline.

A chunk of text. Emitted by LLM services, consumed by context aggregators, TTS services and more. Can be used to send text through processors.

Parameters:

• text – The text content.

• skip_tts – Whether this text should be skipped by the TTS service.

• includes_inter_frame_spaces – Whether any necessary inter-frame (leading/trailing) spaces are already included in the text.

• append_to_context – Whether this text should be appended to the LLM context. Defaults to True.

class pipecat.frames.frames.LLMTextFrame(text: str)

Bases: TextFrame

Text frame generated by LLM services.

class pipecat.frames.frames.LLMMarkerFrame(marker: str, append_to_context_immediately: bool = True)

Bases: DataFrame

Sideband marker emitted by an LLM service.

A marker is short, structured assistant output that should be persisted in the conversation context but should not flow through the standard text path (TTS, transcript). The assistant aggregator writes the marker to the context so the LLM can self-condition on prior markers on subsequent turns.

The primary use today is the filter_incomplete_user_turns protocol, where UserTurnCompletionLLMServiceMixin emits the turn-completion markers ● / ◐ / ○ on every response. The frame is intentionally generic so other components — STT services with built-in turn signals, end-of-turn classifiers, custom annotations, etc. — can use the same mechanism to inject sideband signals into the assistant context.

Parameters:

• marker – The marker payload (typically a short string such as a single character).

• append_to_context_immediately – If True, the marker is written to the context as its own standalone assistant message as soon as it’s received. If False, the marker is appended to the running assistant aggregation and flushed to the context together with the following text as a single message (e.g. for the ● case the context message ends up as “● <response>”).

append_to_context_immediately: bool = True

class pipecat.frames.frames.AggregatedTextFrame(text: str, aggregated_by: AggregationType | str, context_id: str | None = None, raw_text: str | None = None)

Bases: TextFrame

Text frame representing an aggregation of TextFrames.

This frame contains multiple TextFrames aggregated together for processing or output along with a field to indicate how they are aggregated.

Parameters:

• aggregated_by – Method used to aggregate the text frames.

• context_id – Unique identifier for the TTS context that generated this text.

• raw_text – The full matched text including start/end pattern delimiters, set when this frame was produced from a PatternMatch (e.g. a <code>...</code> block). None for ordinary sentence aggregations.

• will_be_spoken – Whether the TTS service will speak this frame. Set to True by the TTS service just before synthesis. Defaults to False.

context_id: str | None = None

raw_text: str | None = None

will_be_spoken: bool = False

class pipecat.frames.frames.VisionTextFrame(text: str)

Bases: LLMTextFrame

Text frame generated by vision services.

class pipecat.frames.frames.TTSTextFrame(text: str, aggregated_by: AggregationType | str, context_id: str | None = None, raw_text: str | None = None)

Bases: AggregatedTextFrame

Text frame generated by Text-to-Speech services.

class pipecat.frames.frames.AggregatedTextProgressFrame(segment_id: int, context_id: str | None, text: str, aggregated_by: AggregationType | str, accumulated_text: str, remaining_text: str)

Bases: DataFrame

Progress frame emitted alongside each TTSTextFrame during word-timestamp playback.

Carries the spoken-so-far / remaining-text breakdown for the active AggregatedTextFrame slot, enabling downstream consumers (e.g. the RTVI observer) to do word-level highlighting without coupling to internal sequencer state.

Parameters:

• segment_id – ID of the AggregatedTextFrame being spoken.

• context_id – TTS context this frame belongs to.

• text – Full original text of the source AggregatedTextFrame.

• aggregated_by – Aggregation type of the source AggregatedTextFrame.

• accumulated_text – Text already spoken in this slot (including the current word).

• remaining_text – Text not yet spoken in this slot.

class pipecat.frames.frames.TranscriptionFrame(text: str, user_id: str, timestamp: str, language: Language | None = None, result: Any | None = None, finalized: bool = False)

Bases: TextFrame

Text frame containing speech transcription data.

A text frame with transcription-specific data. The result field contains the result from the STT service if available.

Parameters:

• user_id – Identifier for the user who spoke.

• timestamp – When the transcription occurred.

• language – Detected or specified language of the speech.

• result – Raw result from the STT service.

• finalized – Whether this is the final transcription for an utterance. Set by STT services that support commit/finalize signals.

language: Language | None = None

result: Any | None = None

finalized: bool = False

class pipecat.frames.frames.InterimTranscriptionFrame(text: str, user_id: str, timestamp: str, language: Language | None = None, result: Any | None = None)

Bases: TextFrame

Text frame containing partial/interim transcription data.

A text frame with interim transcription-specific data that represents partial results before final transcription. The result field contains the result from the STT service if available.

Parameters:

• user_id – Identifier for the user who spoke.

• timestamp – When the interim transcription occurred.

• language – Detected or specified language of the speech.

• result – Raw result from the STT service.

language: Language | None = None

result: Any | None = None

class pipecat.frames.frames.TranslationFrame(text: str, user_id: str, timestamp: str, language: Language | None = None)

Bases: TextFrame

Text frame containing translated transcription data.

A text frame with translated transcription data that will be placed in the transport’s receive queue when a participant speaks.

Parameters:

• user_id – Identifier for the user who spoke.

• timestamp – When the translation occurred.

• language – Target language of the translation.

language: Language | None = None

class pipecat.frames.frames.LLMContextAssistantTimestampFrame(timestamp: str)

Bases: DataFrame

Timestamp information for assistant messages in LLM context.

Parameters:

timestamp – Timestamp when the assistant message was created.

class pipecat.frames.frames.LLMContextAssistantTurnFrame(text: str, timestamp: str)

Bases: DataFrame

The aggregated text of a completed assistant turn.

Broadcast by the LLM assistant aggregator when a turn ends, carrying the same text that is stored in the LLM context. Processors upstream and downstream (e.g. STT services) can handle this frame to react to each completed bot reply without needing a separate observer.

Parameters:

• text – The assistant’s aggregated spoken text for this turn.

• timestamp – ISO-8601 timestamp of when the assistant turn started.

class pipecat.frames.frames.LLMContextFrame(context: LLMContext)

Bases: Frame

Frame containing a universal LLM context.

Used as a signal to LLM services to ingest the provided context and generate a response based on it.

Parameters:

context – The LLM context containing messages, tools, and configuration.

class pipecat.frames.frames.LLMThoughtStartFrame(append_to_context: bool = False, llm: str | None = None)

Bases: ControlFrame

Frame indicating the start of an LLM thought.

Parameters:

• append_to_context – Whether the thought should be appended to the LLM context. If it is appended, the llm field is required, since it will be appended as an LLMSpecificMessage.

• llm – Optional identifier of the LLM provider for LLM-specific handling. Only required if append_to_context is True, as the thought is appended to context as an LLMSpecificMessage.

append_to_context: bool = False

llm: str | None = None

class pipecat.frames.frames.LLMThoughtTextFrame(text: str)

Bases: DataFrame

Frame containing the text (or text chunk) of an LLM thought.

Note that despite this containing text, it is a DataFrame and not a TextFrame, to avoid most typical text processing, such as TTS.

Parameters:

text – The text (or text chunk) of the thought.

includes_inter_frame_spaces: bool

class pipecat.frames.frames.LLMThoughtEndFrame(signature: Any = None)

Bases: ControlFrame

Frame indicating the end of an LLM thought.

Parameters:

signature – Optional signature associated with the thought. This is used by Anthropic, which includes a signature at the end of each thought.

signature: Any = None

class pipecat.frames.frames.LLMRunFrame

Bases: DataFrame

Frame to trigger LLM processing with current context.

A frame that instructs the LLM service to process the current context and generate a response.

class pipecat.frames.frames.LLMMessagesAppendFrame(messages: list[LLMContextMessage], run_llm: bool | None = None)

Bases: DataFrame

Frame containing LLM messages to append to current context.

A frame containing a list of LLM messages that need to be added to the current context.

Parameters:

• messages – List of context messages to append.

• run_llm – Whether the context update should be sent to the LLM.

run_llm: bool | None = None

class pipecat.frames.frames.LLMMessagesUpdateFrame(messages: list[LLMContextMessage], run_llm: bool | None = None)

Bases: DataFrame

Frame containing LLM messages to replace current context.

A frame containing a list of new LLM messages to replace the current context LLM messages.

Parameters:

• messages – List of context messages to replace current context.

• run_llm – Whether the context update should be sent to the LLM.

run_llm: bool | None = None

class pipecat.frames.frames.LLMMessagesTransformFrame(transform: Callable[[list[LLMContextMessage]], list[LLMContextMessage]], run_llm: bool | None = None)

Bases: DataFrame

Frame containing a transform function to modify the current context’s LLM messages.

A frame containing a transform function that takes the context’s current list of LLM messages and returns a modified list.

Parameters:

• transform – A function that takes a list of messages and returns a modified list.

• run_llm – Whether the context update should be sent to the LLM.

run_llm: bool | None = None

class pipecat.frames.frames.LLMSetToolsFrame(tools: list[FunctionSchema | DirectFunction] | ToolsSchema | NotGiven)

Bases: DataFrame

Frame containing tools for LLM function calling.

Used to change the set of tools advertised to the LLM mid-conversation.

Parameters:

tools – The tools to advertise. May be a ToolsSchema, a plain list of direct functions and/or FunctionSchema objects (normalized to a ToolsSchema, with direct-function handlers auto-registered), or NOT_GIVEN to clear tools. Provider-native tools travel in a ToolsSchema’s custom_tools, keyed by adapter type.

class pipecat.frames.frames.LLMSetToolChoiceFrame(tool_choice: Literal['none', 'auto', 'required'] | dict)

Bases: DataFrame

Frame containing tool choice configuration for LLM function calling.

Parameters:

tool_choice – Tool choice setting - ‘none’, ‘auto’, ‘required’, or specific tool dict.

class pipecat.frames.frames.LLMEnablePromptCachingFrame(enable: bool)

Bases: DataFrame

Frame to enable/disable prompt caching in LLMs.

Parameters:

enable – Whether to enable prompt caching.

class pipecat.frames.frames.LLMConfigureOutputFrame(skip_tts: bool)

Bases: DataFrame

Frame to configure LLM output.

This frame is used to configure how the LLM produces output. For example, it can tell the LLM to generate tokens that should be added to the context but not spoken by the TTS service (if one is present in the pipeline).

Parameters:

skip_tts – Whether LLM tokens should skip the TTS service (if any).

class pipecat.frames.frames.FunctionCallResultProperties(run_llm: bool | None = None, on_context_updated: Callable[[], Awaitable[None]] | None = None, is_final: bool = True)

Bases: object

Properties for configuring function call result behavior.

Parameters:

• run_llm – Whether to run the LLM after receiving this result.

• on_context_updated – Callback to execute when context is updated.

• is_final – Whether this is the final result for the function call. When False the result is treated as an intermediate update. Defaults to True. Only meaningful for async function calls (cancel_on_interruption=False). Note: realtime LLM services do not support streamed intermediate results; they deliver only the final result to the provider. An intermediate result reported to a realtime service is dropped and an error is raised. Use a non-realtime LLM service if your tool needs to stream intermediate results.

run_llm: bool | None = None

on_context_updated: Callable[[], Awaitable[None]] | None = None

is_final: bool = True

class pipecat.frames.frames.FunctionCallResultFrame(function_name: str, tool_call_id: str, arguments: Any, result: Any, run_llm: bool | None = None, properties: FunctionCallResultProperties | None = None)

Bases: DataFrame, UninterruptibleFrame

Frame containing the result of an LLM function call.

This is an uninterruptible frame because once a result is generated we always want to update the context.

Parameters:

• function_name – Name of the function that was executed.

• tool_call_id – Unique identifier for the function call.

• arguments – Arguments that were passed to the function.

• result – The result returned by the function.

• run_llm – Whether to run the LLM after this result.

• properties – Additional properties for result handling.

run_llm: bool | None = None

properties: FunctionCallResultProperties | None = None

class pipecat.frames.frames.TTSSpeakFrame(text: str, append_to_context: bool = True)

Bases: DataFrame

Frame containing text that should be spoken by TTS.

A frame that contains text that should be spoken by the TTS service in the pipeline (if any).

Parameters:

• text – The text to be spoken.

• append_to_context – Whether the spoken text should be appended to the LLM context. Defaults to True. (Note that, as of version 1.4.0, None — the previous default — is no longer a supported value.)

append_to_context: bool = True

class pipecat.frames.frames.OutputTransportMessageFrame(message: Any)

Bases: DataFrame

Frame containing transport-specific message data.

Parameters:

message – The transport message payload.

class pipecat.frames.frames.DTMFFrame

Bases: object

Marker base class for DTMF (Dual-Tone Multi-Frequency) keypad frames.

Used only as a shared tag so that both input and output DTMF frames can be identified via isinstance(frame, DTMFFrame). The concrete frames define their own fields.

class pipecat.frames.frames.OutputDTMFFrame(button: KeypadEntry | None = None, buttons: list[KeypadEntry] | None = None)

Bases: DTMFFrame, DataFrame

DTMF keypress output frame for transport queuing.

Parameters:

• button – Convenience shortcut for sending a single DTMF keypad entry. Equivalent to buttons=[button]. If both buttons and button are provided, buttons takes precedence.

• buttons – Sequence of one or more DTMF keypad buttons to send. Use from_string() to build this from a string like "123#".

button: KeypadEntry | None = None

buttons: list[KeypadEntry] | None = None

classmethod from_string(buttons: str, **kwargs) → OutputDTMFFrame

Build an OutputDTMFFrame from a string of DTMF characters.

Parameters:

• buttons – A string like "123#". Each character must be a valid KeypadEntry value.

• **kwargs – Additional keyword arguments forwarded to the frame constructor.

Returns:

A frame of type cls with buttons populated as a list of KeypadEntry.

to_string() → str

Return the frame’s buttons as a dial string.

Returns:

A string such as "123#" formed by concatenating the values of each KeypadEntry in buttons, or an empty string if buttons is not set.

class pipecat.frames.frames.StartFrame(audio_in_sample_rate: int = 16000, audio_out_sample_rate: int = 24000, enable_metrics: bool = False, enable_tracing: bool = False, enable_usage_metrics: bool = False, report_only_initial_ttfb: bool = False, tracing_context: TracingContext | None = None)

Bases: SystemFrame

Initial frame to start pipeline processing.

This is the first frame that should be pushed down a pipeline to initialize all processors with their configuration parameters.

Parameters:

• audio_in_sample_rate –

  Input audio sample rate in Hz.

  Deprecated since version 1.8.0: Read audio_in_sample_rate in FrameProcessorSetup.setup() instead. Will be removed in 2.0.0.

• audio_out_sample_rate –

  Output audio sample rate in Hz.

  Deprecated since version 1.8.0: Read audio_out_sample_rate in FrameProcessorSetup.setup() instead. Will be removed in 2.0.0.

• enable_metrics –

  Whether to enable performance metrics collection.

  Deprecated since version 1.8.0: Read enable_metrics in FrameProcessorSetup.setup() instead. Will be removed in 2.0.0.

• enable_tracing –

  Whether to enable OpenTelemetry tracing.

  Deprecated since version 1.8.0: Read enable_tracing in FrameProcessorSetup.setup() instead. Will be removed in 2.0.0.

• enable_usage_metrics –

  Whether to enable usage metrics collection.

  Deprecated since version 1.8.0: Read enable_usage_metrics in FrameProcessorSetup.setup() instead. Will be removed in 2.0.0.

• report_only_initial_ttfb –

  Whether to report only initial time-to-first-byte.

  Deprecated since version 1.8.0: Read report_only_initial_ttfb in FrameProcessorSetup.setup() instead. Will be removed in 2.0.0.

• tracing_context –

  Pipeline-scoped tracing context for span hierarchy.

  Deprecated since version 1.8.0: Read tracing_context in FrameProcessorSetup.setup() instead. Will be removed in 2.0.0.

audio_in_sample_rate: int = 16000

audio_out_sample_rate: int = 24000

enable_metrics: bool = False

enable_tracing: bool = False

enable_usage_metrics: bool = False

report_only_initial_ttfb: bool = False

tracing_context: TracingContext | None = None

class pipecat.frames.frames.CancelFrame(reason: Any | None = None)

Bases: SystemFrame

Frame indicating pipeline should stop immediately.

Indicates that a pipeline needs to stop right away without processing remaining queued frames.

Parameters:

reason – Optional reason for pushing a cancel frame.

reason: Any | None = None

class pipecat.frames.frames.ErrorFrame(error: str, fatal: bool = False, processor: FrameProcessor | None = None, exception: Exception | None = None, category: ErrorCategory | None = None)

Bases: SystemFrame

Frame notifying of errors in the pipeline.

This is used to notify upstream that an error has occurred downstream in the pipeline.

Parameters:

• error – Description of the error that occurred.

• fatal –

  Whether the error is fatal and requires bot shutdown.

  Deprecated since version 1.8.0: Use FrameProcessor.push_error() with force_treat_as_permanent=True instead, when the error leaves its originating processor unable to do its job: it marks that processor unusable and the pipeline worker applies its ProcessorUnusablePolicy. For an error that isn’t about a processor’s state, push an EndWorkerFrame after the error to end the pipeline. Will be removed in 2.0.0.

• processor – The frame processor that generated the error.

• exception – The exception that occurred.

• category – What kind of failure this was, drawn from ErrorCategory: rejected credentials, an unreachable provider, a malformed request and so on. None means nobody has said yet, which invites the category to be worked out from the exception; set it to ErrorCategory.UNKNOWN to report an error whose cause can’t be attributed. Always set by the time the frame travels.

fatal: bool = False

processor: FrameProcessor | None = None

exception: Exception | None = None

category: ErrorCategory | None = None

class pipecat.frames.frames.FatalErrorFrame(**kwargs)

Bases: ErrorFrame

Frame notifying of unrecoverable errors requiring bot shutdown.

Deprecated since version 1.8.0: Use ErrorFrame instead. Report the error with FrameProcessor.push_error() and force_treat_as_permanent=True when it leaves its originating processor unable to do its job — the pipeline worker then applies its ProcessorUnusablePolicy, of which CANCEL shuts the bot down. For an error that isn’t about a processor’s state, push an EndWorkerFrame after the error instead. Will be removed in 2.0.0.

Parameters:

fatal – Always True for fatal errors.

fatal: bool = True

class pipecat.frames.frames.FrameProcessorPauseUrgentFrame(processor: FrameProcessor)

Bases: SystemFrame

Frame to pause frame processing immediately.

This frame is used to pause frame processing for the given processor as fast as possible. Pausing frame processing will keep frames in the internal queue which will then be processed when frame processing is resumed with FrameProcessorResumeFrame.

Parameters:

processor – The frame processor to pause.

class pipecat.frames.frames.FrameProcessorResumeUrgentFrame(processor: FrameProcessor)

Bases: SystemFrame

Frame to resume frame processing immediately.

This frame is used to resume frame processing for the given processor if it was previously paused as fast as possible. After resuming frame processing all queued frames will be processed in the order received.

Note

This frame is now equivalent to FrameProcessorResumeFrame, which was changed to a SystemFrame to fix a bug where resume frames would get stuck in the blocked processing queue.

Parameters:

processor – The frame processor to resume.

class pipecat.frames.frames.InterruptionFrame

Bases: SystemFrame

Frame pushed to interrupt the pipeline.

This frame is used to interrupt the pipeline. For example, when a user starts speaking to cancel any in-progress bot output. It can also be pushed by any processor.

class pipecat.frames.frames.UserStartedSpeakingFrame

Bases: SystemFrame

Frame indicating that the user turn has started.

Emitted when the user turn starts, which usually means that some transcriptions are already available.

class pipecat.frames.frames.UserStoppedSpeakingFrame

Bases: SystemFrame

Frame indicating that the user turn has ended.

Emitted when the user turn ends. This usually coincides with the start of the bot turn.

class pipecat.frames.frames.UserMuteStartedFrame

Bases: SystemFrame

Frame indicating that the user has been muted.

Emitted when a mute strategy activates, suppressing user frames (audio, transcription, interruption) from propagating through the pipeline.

class pipecat.frames.frames.UserMuteStoppedFrame

Bases: SystemFrame

Frame indicating that the user has been unmuted.

Emitted when a mute strategy deactivates, allowing user frames to propagate through the pipeline again.

class pipecat.frames.frames.UserSpeakingFrame

Bases: SystemFrame

Frame indicating the user is speaking.

Emitted by VAD to indicate the user is speaking.

class pipecat.frames.frames.UserTurnInferenceCompletedFrame

Bases: SystemFrame

Frame indicating that the user turn is semantically complete.

Emitted by any component that can judge conversational turn completeness — for example an LLM with turn-completion markers, an STT service with built-in turn detection, or a dedicated end-of-turn classifier. Stop strategies that gate the user-turn-stop event on an external completeness signal (e.g. LLMTurnCompletionUserTurnStopStrategy) consume this frame to finalize the turn. Producers should emit this frame only when they judge the turn complete; an absence of this frame means the turn is not yet considered complete.

class pipecat.frames.frames.VADUserStartedSpeakingFrame(start_secs: float = 0.0, timestamp: float = <factory>)

Bases: SystemFrame

Frame emitted when VAD definitively detects user started speaking.

Parameters:

• start_secs – The VAD start_secs duration that was used to confirm the user started speaking. This represents the speech duration that had to elapse before the VAD determined speech began.

• timestamp – Wall-clock time when the VAD made its determination.

start_secs: float = 0.0

class pipecat.frames.frames.VADUserStoppedSpeakingFrame(stop_secs: float = 0.0, timestamp: float = <factory>)

Bases: SystemFrame

Frame emitted when VAD definitively detects user stopped speaking.

Parameters:

• stop_secs – The VAD stop_secs duration that was used to confirm the user stopped speaking. This represents the silence duration that had to elapse before the VAD determined speech ended.

• timestamp – Wall-clock time when the VAD made its determination.

stop_secs: float = 0.0

class pipecat.frames.frames.ProposedUserStartedSpeakingFrame

Bases: SystemFrame

Frame proposing that the user turn has started.

Emitted by a component with its own turn detection — typically an STT or realtime LLM service whose provider reports speech boundaries. It is a proposal, not a decision: an ExternalUserTurnStartStrategy resolves it into a UserStartedSpeakingFrame and broadcasts the interruption.

This is a system frame because resolving it broadcasts an interruption, which must preempt queued frames rather than wait behind them. Its end-of-turn counterpart has the opposite requirement and is a control frame; see ProposedUserStoppedSpeakingFrame.

class pipecat.frames.frames.ProposedUserStoppedSpeakingFrame

Bases: ControlFrame

Frame proposing that the user turn has ended.

The end-of-turn counterpart to ProposedUserStartedSpeakingFrame, resolved into a UserStoppedSpeakingFrame by an ExternalUserTurnStopStrategy.

This is a control frame so it stays ordered against the final TranscriptionFrame. A service with its own turn detection pushes that transcript and then proposes the stop, and the turn strategy needs that text in hand to close the turn on.

class pipecat.frames.frames.BotStartedSpeakingFrame

Bases: SystemFrame

Frame indicating the bot started speaking.

Emitted upstream and downstream by the BaseTransportOutput to indicate the bot started speaking.

class pipecat.frames.frames.BotStoppedSpeakingFrame

Bases: SystemFrame

Frame indicating the bot stopped speaking.

Emitted upstream and downstream by the BaseTransportOutput to indicate the bot stopped speaking.

class pipecat.frames.frames.BotSpeakingFrame

Bases: SystemFrame

Frame indicating the bot is currently speaking.

Emitted upstream and downstream by the BaseOutputTransport while the bot is still speaking. This can be used, for example, to detect when a user is idle. That is, while the bot is speaking we don’t want to trigger any user idle timeout since the user might be listening.

class pipecat.frames.frames.MetricsFrame(data: list[MetricsData])

Bases: SystemFrame

Frame containing performance metrics data.

Emitted by processors that can compute metrics like latencies.

Parameters:

data – List of metrics data collected by the processor.

class pipecat.frames.frames.FunctionCallFromLLM(function_name: str, tool_call_id: str, arguments: Mapping[str, Any], context: Any)

Bases: object

Represents a function call returned by the LLM.

Represents a function call returned by the LLM to be registered for execution.

Parameters:

• function_name – The name of the function to call.

• tool_call_id – A unique identifier for the function call.

• arguments – The arguments to pass to the function.

• context – The LLM context when the function call was made.

class pipecat.frames.frames.FunctionCallsStartedFrame(function_calls: Sequence[FunctionCallFromLLM])

Bases: SystemFrame

Frame signaling that function call execution is starting.

A frame signaling that one or more function call execution is going to start.

Parameters:

function_calls – Sequence of function calls that will be executed.

class pipecat.frames.frames.FunctionCallCancelFrame(function_name: str, tool_call_id: str, run_llm: bool = False)

Bases: SystemFrame

Frame signaling that a function call has been cancelled.

Parameters:

• function_name – Name of the function that was cancelled.

• tool_call_id – Unique identifier for the cancelled function call.

• run_llm – Whether to run the LLM after this cancellation. Only a call cancelled by its own timeout sets this: an interruption must not trigger inference, and a cancellation the LLM asked for already runs inference through the result of the tool that requested it.

run_llm: bool = False

class pipecat.frames.frames.STTMuteFrame(mute: bool)

Bases: SystemFrame

Frame to mute/unmute the Speech-to-Text service.

Parameters:

mute – Whether to mute (True) or unmute (False) the STT service.

class pipecat.frames.frames.InputTransportMessageFrame(message: Any)

Bases: SystemFrame

Frame for transport messages received from external sources.

Parameters:

message – The urgent transport message payload.

class pipecat.frames.frames.OutputTransportMessageUrgentFrame(message: Any)

Bases: SystemFrame

Frame for urgent transport messages that need to be sent immediately.

Parameters:

message – The urgent transport message payload.

class pipecat.frames.frames.UserImageRequestFrame(user_id: str, text: str | None = None, append_to_context: bool | None = None, video_source: str | None = None, function_name: str | None = None, tool_call_id: str | None = None, result_callback: Any | None = None)

Bases: SystemFrame

Frame requesting an image from a specific user.

A frame to request an image from the given user. The request might come with a text that can be later used to describe the requested image.

Parameters:

• user_id – Identifier of the user to request image from.

• text – An optional text associated to the image request.

• append_to_context – Whether the requested image should be appended to the LLM context.

• video_source – Specific video source to capture from.

• function_name – Name of function that generated this request (if any).

• tool_call_id – Tool call ID if generated by function call (if any).

• result_callback – Optional callback to invoke when the image is retrieved.

text: str | None = None

append_to_context: bool | None = None

video_source: str | None = None

function_name: str | None = None

tool_call_id: str | None = None

result_callback: Any | None = None

class pipecat.frames.frames.InputAudioRawFrame(audio: bytes, sample_rate: int, num_channels: int)

Bases: SystemFrame, AudioRawFrame

Raw audio input frame from transport.

A chunk of audio usually coming from an input transport. If the transport supports multiple audio sources (e.g. multiple audio tracks) the source name will be specified in transport_source.

class pipecat.frames.frames.InputImageRawFrame(image: bytes, size: tuple[int, int], format: str | None)

Bases: SystemFrame, ImageRawFrame

Raw image input frame from transport.

An image usually coming from an input transport. If the transport supports multiple video sources (e.g. multiple video tracks) the source name will be specified in transport_source.

class pipecat.frames.frames.InputTextRawFrame(text: str)

Bases: SystemFrame, TextFrame

Raw text input frame from transport.

Text input usually coming from user typing or programmatic text injection that should be sent to LLM services as input, similar to how InputAudioRawFrame and InputImageRawFrame represent user audio and video input.

class pipecat.frames.frames.UserAudioRawFrame(audio: bytes, sample_rate: int, num_channels: int, user_id: str = '')

Bases: InputAudioRawFrame

Raw audio input frame associated with a specific user.

A chunk of audio, usually coming from an input transport, associated to a user.

Parameters:

user_id – Identifier of the user who provided this audio.

user_id: str = ''

class pipecat.frames.frames.UserImageRawFrame(image: bytes, size: tuple[int, int], format: str | None, user_id: str = '', text: str | None = None, append_to_context: bool | None = None, request: UserImageRequestFrame | None = None)

Bases: InputImageRawFrame

Raw image input frame associated with a specific user.

An image associated to a user, potentially in response to an image request.

Parameters:

• user_id – Identifier of the user who provided this image.

• text – An optional text associated to this image.

• append_to_context – Whether the requested image should be appended to the LLM context.

• request – The original image request frame if this is a response.

user_id: str = ''

text: str | None = None

append_to_context: bool | None = None

request: UserImageRequestFrame | None = None

class pipecat.frames.frames.AssistantImageRawFrame(image: bytes, size: tuple[int, int], format: str | None, original_data: bytes | None = None, original_mime_type: str | None = None)

Bases: OutputImageRawFrame

Frame containing an image generated by the assistant.

Contains both the raw frame for display (superclass functionality) as well as the original image, which can get used directly in LLM contexts.

Parameters:

• original_data – The original image data, which can get used directly in an LLM context message without further encoding.

• original_mime_type – The MIME type of the original image data.

original_data: bytes | None = None

original_mime_type: str | None = None

class pipecat.frames.frames.InputDTMFFrame(button: KeypadEntry)

Bases: DTMFFrame, SystemFrame

DTMF keypress input frame from transport.

Parameters:

button – The DTMF keypad entry that was pressed.

class pipecat.frames.frames.OutputDTMFUrgentFrame(button: KeypadEntry | None = None, buttons: list[KeypadEntry] | None = None)

Bases: DTMFFrame, SystemFrame

DTMF keypress output frame for immediate sending.

Parameters:

• button – Convenience shortcut for sending a single DTMF keypad entry. Equivalent to buttons=[button]. If both buttons and button are provided, buttons takes precedence.

• buttons – Sequence of one or more DTMF keypad buttons to send. Use from_string() to build this from a string like "123#".

button: KeypadEntry | None = None

buttons: list[KeypadEntry] | None = None

classmethod from_string(buttons: str, **kwargs) → OutputDTMFUrgentFrame

Build an OutputDTMFUrgentFrame from a string of DTMF characters.

Parameters:

• buttons – A string like "123#". Each character must be a valid KeypadEntry value.

• **kwargs – Additional keyword arguments forwarded to the frame constructor.

Returns:

A frame of type cls with buttons populated as a list of KeypadEntry.

to_string() → str

Return the frame’s buttons as a dial string.

Returns:

A string such as "123#" formed by concatenating the values of each KeypadEntry in buttons, or an empty string if buttons is not set.

class pipecat.frames.frames.SpeechControlParamsFrame(vad_params: VADParams | None = None, turn_params: BaseTurnParams | None = None)

Bases: SystemFrame

Frame for notifying processors of speech control parameter changes.

This includes parameters for both VAD (Voice Activity Detection) and turn-taking analysis. It allows downstream processors to adjust their behavior based on updated interaction control settings.

Parameters:

• vad_params – Current VAD parameters.

• turn_params – Current turn-taking analysis parameters.

vad_params: VADParams | None = None

turn_params: BaseTurnParams | None = None

class pipecat.frames.frames.ServiceMetadataFrame(service_name: str, *, user_turn_strategies: UserTurnStrategies | None = None)

Bases: SystemFrame

Base metadata frame for services.

Broadcast by services at pipeline start to share service-specific configuration and performance characteristics with downstream processors.

Parameters:

• service_name – The name of the service broadcasting this metadata.

• user_turn_strategies – User turn strategies the service recommends, e.g. ExternalUserTurnStrategies for a service that does its own server-side end-of-turn detection. The user aggregator applies them unless the user passed their own user_turn_strategies, which always wins. None leaves the defaults in place.

user_turn_strategies: UserTurnStrategies | None = None

class pipecat.frames.frames.STTMetadataFrame(service_name: str, ttfs_p99_latency: float, *, user_turn_strategies: UserTurnStrategies | None = None)

Bases: ServiceMetadataFrame

Metadata from STT service.

Broadcast by STT services to inform downstream processors (like turn strategies) about STT latency characteristics.

Parameters:

ttfs_p99_latency – Time to final segment P99 latency in seconds. This is the expected time from when speech ends to when the final transcript is received, at the 99th percentile.

class pipecat.frames.frames.LLMServiceMetadataFrame(service_name: str, is_realtime_service: bool = False, *, user_turn_strategies: UserTurnStrategies | None = None)

Bases: ServiceMetadataFrame

Metadata broadcast by an LLM service at pipeline start.

Parameters:

is_realtime_service – Whether the broadcasting service is a realtime (speech-to-speech) LLM service.

is_realtime_service: bool = False

class pipecat.frames.frames.ServiceSwitcherRequestMetadataFrame(service: FrameProcessor)

Bases: ControlFrame

Request a service to re-emit its metadata frames.

Used by ServiceSwitcher when switching active services to ensure downstream processors receive updated metadata from the newly active service. Services that receive this frame should re-push their metadata frame (e.g., STTMetadataFrame for STT services).

Parameters:

service – The target service that should re-emit its metadata.

class pipecat.frames.frames.WorkerFrame

Bases: ControlFrame

Base frame for worker frames.

This is a base class for frames that are handled by the pipeline worker. This might result in a corresponding frame sent downstream (e.g. InterruptionWorkerFrame / InterruptionFrame or EndWorkerFrame / EndFrame). Push these frames downstream (the default direction) so frames queued ahead of them are processed first; pushing them upstream is also supported.

class pipecat.frames.frames.WorkerSystemFrame

Bases: SystemFrame

Base frame for worker system frames.

This is a base class for system frames that are handled by the pipeline worker. This might result in a corresponding frame sent downstream (e.g. InterruptionWorkerFrame / InterruptionFrame or CancelWorkerFrame / CancelFrame). They can be pushed in either direction (downstream is the default); being system frames, they are processed immediately either way.

class pipecat.frames.frames.EndWorkerFrame(reason: Any | None = None)

Bases: WorkerFrame, UninterruptibleFrame

Frame to request graceful pipeline worker closure.

This is used to notify the pipeline worker that the pipeline should be closed nicely (flushing all the queued frames) by pushing an EndFrame downstream. Push this frame downstream (the default direction) so frames queued ahead of it are flushed before the pipeline ends.

Parameters:

reason – Optional reason for pushing an end frame.

reason: Any | None = None

class pipecat.frames.frames.StopWorkerFrame

Bases: WorkerFrame, UninterruptibleFrame

Frame to request pipeline worker stop while keeping processors running.

This is used to notify the pipeline worker that it should be stopped as soon as possible (flushing all the queued frames) but that the pipeline processors should be kept in a running state. Push this frame downstream (the default direction) so frames queued ahead of it are flushed first.

class pipecat.frames.frames.CancelWorkerFrame(reason: Any | None = None)

Bases: WorkerSystemFrame

Frame to request immediate pipeline worker cancellation.

This is used to notify the pipeline worker that the pipeline should be stopped immediately by pushing a CancelFrame downstream. It can be pushed in either direction; being a system frame, it is processed immediately.

Parameters:

reason – Optional reason for pushing a cancel frame.

reason: Any | None = None

class pipecat.frames.frames.InterruptionWorkerFrame

Bases: WorkerSystemFrame

Frame indicating the pipeline should be interrupted.

The pipeline worker converts this into an InterruptionFrame and sends it downstream. It can be pushed in either direction; being a system frame, it is processed immediately.

class pipecat.frames.frames.TaskFrame(**kwargs)

Bases: WorkerFrame

Deprecated alias for WorkerFrame.

Deprecated since version 1.4.0: Use WorkerFrame instead. Will be removed in 2.0.0.

class pipecat.frames.frames.TaskSystemFrame(**kwargs)

Bases: WorkerSystemFrame

Deprecated alias for WorkerSystemFrame.

Deprecated since version 1.4.0: Use WorkerSystemFrame instead. Will be removed in 2.0.0.

class pipecat.frames.frames.EndTaskFrame(**kwargs)

Bases: EndWorkerFrame, TaskFrame

Deprecated alias for EndWorkerFrame.

Deprecated since version 1.4.0: Use EndWorkerFrame instead. Will be removed in 2.0.0.

class pipecat.frames.frames.StopTaskFrame(**kwargs)

Bases: StopWorkerFrame, TaskFrame

Deprecated alias for StopWorkerFrame.

Deprecated since version 1.4.0: Use StopWorkerFrame instead. Will be removed in 2.0.0.

class pipecat.frames.frames.CancelTaskFrame(**kwargs)

Bases: CancelWorkerFrame, TaskSystemFrame

Deprecated alias for CancelWorkerFrame.

Deprecated since version 1.4.0: Use CancelWorkerFrame instead. Will be removed in 2.0.0.

class pipecat.frames.frames.InterruptionTaskFrame(**kwargs)

Bases: InterruptionWorkerFrame, TaskSystemFrame

Deprecated alias for InterruptionWorkerFrame.

Deprecated since version 1.4.0: Use InterruptionWorkerFrame instead. Will be removed in 2.0.0.

class pipecat.frames.frames.EndFrame(reason: Any | None = None)

Bases: ControlFrame, UninterruptibleFrame

Frame indicating pipeline has ended and should shut down.

Indicates that a pipeline has ended and frame processors and pipelines should be shut down. If the transport receives this frame, it will stop sending frames to its output channel(s) and close all its threads. Note, that this is a control frame, which means it will be received in the order it was sent.

This frame is marked as UninterruptibleFrame to ensure it is not lost when an InterruptionFrame is processed. Terminal frames must survive interruption to guarantee proper pipeline shutdown.

Parameters:

reason – Optional reason for pushing an end frame.

reason: Any | None = None

class pipecat.frames.frames.StopFrame

Bases: ControlFrame, UninterruptibleFrame

Frame indicating pipeline should stop but keep processors running.

Indicates that a pipeline should be stopped but that the pipeline processors should be kept in a running state. This is normally queued from the pipeline task.

This frame is marked as UninterruptibleFrame to ensure it is not lost when an InterruptionFrame is processed. Terminal frames must survive interruption to guarantee proper pipeline control.

class pipecat.frames.frames.PipelineFlushFrame(event: Event | None = None, returning: bool = False, origin: str | None = None)

Bases: ControlFrame, UninterruptibleFrame

Probe frame used to flush all in-flight frames from the pipeline.

Pushed downstream; the pipeline worker’s sink bounces it back upstream, the source turns it around, and the worker sets event when it reaches the sink a second time. Once that fires, every frame queued ahead of the probe has been processed, along with anything a processor started by pushing upstream. Useful to wait for the pipeline to drain (e.g. after an interruption) before injecting a new frame.

This frame is marked as UninterruptibleFrame so the probe survives an InterruptionFrame and still completes its trip.

Parameters:

• event – Set by the worker when the probe completes its round-trip. The initiator awaits it to know the pipeline has drained. Carried on the frame so concurrent flushes stay isolated (each awaits its own).

• returning – Whether the probe is on its second pass downstream, after having been back up to the source. Work a processor starts by pushing upstream — an LLM run triggered by a function call result, say — only reaches the sink after that turnaround, so the probe makes the trip twice and settles on the second arrival.

• origin – Name of the worker that started the flush. A probe that crosses into another pipeline is answered there, out of sight of whoever is waiting, so the answering worker reports progress back to this name.

event: Event | None = None

returning: bool = False

origin: str | None = None

class pipecat.frames.frames.BotConnectedFrame

Bases: SystemFrame

Frame indicating the bot has connected to the transport service.

Pushed downstream by SFU transports (Daily, LiveKit, HeyGen, Tavus) when the bot successfully joins the room. Non-SFU transports do not emit this frame.

class pipecat.frames.frames.ClientConnectedFrame

Bases: SystemFrame

Frame indicating that a client has connected to the transport.

Pushed downstream by the input transport when a client (participant) connects. Used by observers to measure transport readiness timing.

class pipecat.frames.frames.OutputTransportReadyFrame

Bases: ControlFrame

Frame indicating that the output transport is ready.

Indicates that the output transport is ready and able to receive frames.

class pipecat.frames.frames.InputTransportStartAudioStreamingFrame

Bases: ControlFrame

Frame asking the input transport to start audio input streaming.

Pushed downstream (e.g. by RTVIProcessor once the client is ready) so a BaseInputTransport begins streaming audio from its source. Replaces calling BaseInputTransport.start_audio_in_streaming() directly, keeping cross-processor communication frame-based.

class pipecat.frames.frames.HeartbeatFrame(timestamp: int)

Bases: ControlFrame

Frame used by pipeline task to monitor pipeline health.

This frame is used by the pipeline task as a mechanism to know if the pipeline is running properly.

Parameters:

timestamp – Timestamp when the heartbeat was generated.

class pipecat.frames.frames.FrameProcessorPauseFrame(processor: FrameProcessor)

Bases: ControlFrame

Frame to pause frame processing for a specific processor.

This frame is used to pause frame processing for the given processor. Pausing frame processing will keep frames in the internal queue which will then be processed when frame processing is resumed with FrameProcessorResumeFrame.

Parameters:

processor – The frame processor to pause.

class pipecat.frames.frames.FrameProcessorResumeFrame(processor: FrameProcessor)

Bases: SystemFrame

Frame to resume frame processing for a specific processor.

This frame is used to resume frame processing for the given processor if it was previously paused. After resuming frame processing all queued frames will be processed in the order received.

This is a SystemFrame to ensure it bypasses the blocked processing queue when the processor is paused. Otherwise, the resume frame would get queued and never processed.

Note

This frame is now equivalent to FrameProcessorResumeUrgentFrame.

Parameters:

processor – The frame processor to resume.

class pipecat.frames.frames.LLMFullResponseStartFrame

Bases: ControlFrame

Frame indicating the beginning of an LLM response.

Used to indicate the beginning of an LLM response. Followed by one or more TextFrames and a final LLMFullResponseEndFrame.

skip_tts: bool | None

class pipecat.frames.frames.LLMFullResponseEndFrame

Bases: ControlFrame

Frame indicating the end of an LLM response.

skip_tts: bool | None

class pipecat.frames.frames.LLMAssistantPushAggregationFrame

Bases: ControlFrame

Frame that forces the LLM assistant aggregator to push its current aggregation to context.

When received by LLMAssistantAggregator, any text that has been accumulated in the aggregation buffer is immediately committed to the conversation context as an assistant message, without waiting for an LLMFullResponseEndFrame.

class pipecat.frames.frames.LLMSummarizeContextFrame(config: LLMContextSummaryConfig | None = None)

Bases: ControlFrame

Frame requesting on-demand context summarization.

Push this frame into the pipeline to trigger a manual context summarization.

Parameters:

config – Optional per-request override for summary generation settings (prompt, token budget, messages to keep). If None, the summarizer’s default LLMContextSummaryConfig is used.

config: LLMContextSummaryConfig | None = None

class pipecat.frames.frames.LLMContextSummaryRequestFrame(request_id: str, context: LLMContext, min_messages_to_keep: int, target_context_tokens: int, summarization_prompt: str, summarization_timeout: float | None = None)

Bases: ControlFrame

Frame requesting context summarization from an LLM service.

Sent by aggregators to LLM services when conversation context needs to be compressed. The LLM service generates a summary of older messages while preserving recent conversation history.

Parameters:

• request_id – Unique identifier to match this request with its response. Used to handle async responses and avoid race conditions.

• context – The full LLM context containing all messages to analyze and summarize.

• min_messages_to_keep – Number of recent messages to preserve uncompressed. These messages will not be included in the summary.

• target_context_tokens – Maximum token size for the generated summary. This value is passed directly to the LLM as the max_tokens parameter when generating the summary text.

• summarization_prompt – System prompt instructing the LLM how to generate the summary.

• summarization_timeout – Maximum time in seconds for the LLM to generate a summary. When None, a default timeout of 120s is applied.

summarization_timeout: float | None = None

class pipecat.frames.frames.LLMContextSummaryResultFrame(request_id: str, summary: str, last_summarized_index: int, error: str | None = None)

Bases: ControlFrame, UninterruptibleFrame

Frame containing the result of context summarization.

Sent by LLM services back to aggregators after generating a summary. Contains the formatted summary message and metadata about what was summarized.

Parameters:

• request_id – Identifier matching the original request. Used to correlate async responses.

• summary – The formatted summary message ready to be inserted into context.

• last_summarized_index – Index (0-based) of the last message that was included in the summary. Messages after this index are preserved.

• error – Error message if summarization failed, None on success.

error: str | None = None

class pipecat.frames.frames.FunctionCallInProgressFrame(function_name: str, tool_call_id: str, arguments: Any, cancel_on_interruption: bool = False, group_id: str | None = None)

Bases: ControlFrame, UninterruptibleFrame

Frame signaling that a function call is currently executing.

This is an uninterruptible frame because we always want to update the context.

Parameters:

• function_name – Name of the function being executed.

• tool_call_id – Unique identifier for this function call.

• arguments – Arguments passed to the function.

• cancel_on_interruption – Whether to cancel this call if interrupted. When False the call is treated as asynchronous: the LLM continues the conversation immediately without waiting for the result, and the result is injected later via a developer message.

• group_id – Identifier shared by all function calls originating from the same LLM response batch. Used to determine when the last call in a group completes so the LLM can be triggered exactly once.

cancel_on_interruption: bool = False

group_id: str | None = None

class pipecat.frames.frames.VisionFullResponseStartFrame

Bases: LLMFullResponseStartFrame

Frame indicating the beginning of a vision model response.

Used to indicate the beginning of a vision model response. Followed by one or more VisionTextFrames and a final VisionFullResponseEndFrame.

class pipecat.frames.frames.VisionFullResponseEndFrame

Bases: LLMFullResponseEndFrame

Frame indicating the end of a Vision model response.

class pipecat.frames.frames.TTSStartedFrame(context_id: str | None = None, append_to_context: bool = True)

Bases: ControlFrame

Frame indicating the beginning of a TTS response.

Used to indicate the beginning of a TTS response. Following TTSAudioRawFrames are part of the TTS response until a TTSStoppedFrame. These frames can be used for aggregating audio frames in a transport to optimize the size of frames sent to the session, without needing to control this in the TTS service.

Parameters:

• context_id – Unique identifier for this TTS context.

• append_to_context – Whether the spoken text for this response will be appended to the LLM context. Mirrors the value carried by the response’s TTSTextFrames.

context_id: str | None = None

append_to_context: bool = True

class pipecat.frames.frames.TTSStoppedFrame(context_id: str | None = None)

Bases: ControlFrame

Frame indicating the end of a TTS response.

Parameters:

context_id – Unique identifier for this TTS context.

context_id: str | None = None

class pipecat.frames.frames.ServiceUpdateSettingsFrame(settings: Mapping[str, Any]=<factory>, delta: TSettings | None = None, service: FrameProcessor | None = None, reach_inactive_services: bool = False)

Bases: ControlFrame, UninterruptibleFrame, Generic[TSettings]

Base frame for updating service settings.

Supports both a settings dict (for backward compatibility) and a delta object. When both are provided, delta takes precedence.

Parameters:

• settings –

  Dictionary of setting name to value mappings.

  Deprecated since version 0.0.104: Use delta with a typed settings object instead. Will be removed in 2.0.0.

• delta – ServiceSettings delta-mode object describing the fields to change.

• service – Optional target service instance. When provided, only that service will apply the settings; other services will forward the frame unchanged.

• reach_inactive_services – Whether these settings should reach every service a ServiceSwitcher manages, rather than only the active one. Set this for provider-neutral settings that must survive a service switch.

delta: TSettings | None = None

service: FrameProcessor | None = None

reach_inactive_services: bool = False

class pipecat.frames.frames.LLMUpdateSettingsFrame(settings: Mapping[str, Any]=<factory>, delta: TSettings | None = None, service: FrameProcessor | None = None, reach_inactive_services: bool = False)

Bases: ServiceUpdateSettingsFrame[LLMSettings]

Frame for updating LLM service settings.

class pipecat.frames.frames.TTSUpdateSettingsFrame(settings: Mapping[str, Any]=<factory>, delta: TSettings | None = None, service: FrameProcessor | None = None, reach_inactive_services: bool = False)

Bases: ServiceUpdateSettingsFrame[TTSSettings]

Frame for updating TTS service settings.

class pipecat.frames.frames.STTUpdateSettingsFrame(settings: Mapping[str, Any]=<factory>, delta: TSettings | None = None, service: FrameProcessor | None = None, reach_inactive_services: bool = False)

Bases: ServiceUpdateSettingsFrame[STTSettings]

Frame for updating STT service settings.

class pipecat.frames.frames.UserIdleTimeoutUpdateFrame(timeout: float)

Bases: SystemFrame

Frame for updating the user idle timeout at runtime.

Setting timeout to 0 disables idle detection. Setting a positive value enables it. Updates apply immediately: a running idle timer restarts with the new duration, and if the bot is waiting for the user to speak the timer is armed right away.

Parameters:

timeout – The new idle timeout in seconds. 0 disables idle detection.

class pipecat.frames.frames.VADParamsUpdateFrame(params: VADParams)

Bases: ControlFrame

Frame for updating VAD parameters.

A control frame containing a request to update VAD params. Intended to be pushed upstream from RTVI processor.

Parameters:

params – New VAD parameters to apply.

class pipecat.frames.frames.FilterControlFrame

Bases: ControlFrame

Base control frame for audio filter operations.

class pipecat.frames.frames.FilterUpdateSettingsFrame(settings: Mapping[str, Any])

Bases: FilterControlFrame

Frame for updating audio filter settings.

Parameters:

settings – Dictionary of filter setting name to value mappings.

class pipecat.frames.frames.FilterEnableFrame(enable: bool)

Bases: FilterControlFrame

Frame for enabling/disabling audio filters at runtime.

Parameters:

enable – Whether to enable (True) or disable (False) the filter.

class pipecat.frames.frames.AudioBufferStartRecordingFrame

Bases: ControlFrame, UninterruptibleFrame

Frame instructing audio buffer processors to start recording.

class pipecat.frames.frames.AudioBufferStopRecordingFrame

Bases: ControlFrame, UninterruptibleFrame

Frame instructing audio buffer processors to stop recording and flush.

class pipecat.frames.frames.MixerControlFrame

Bases: ControlFrame

Base control frame for audio mixer operations.

class pipecat.frames.frames.MixerUpdateSettingsFrame(settings: Mapping[str, Any])

Bases: MixerControlFrame

Frame for updating audio mixer settings.

Parameters:

settings – Dictionary of mixer setting name to value mappings.

class pipecat.frames.frames.MixerEnableFrame(enable: bool)

Bases: MixerControlFrame

Frame for enabling/disabling audio mixer at runtime.

Parameters:

enable – Whether to enable (True) or disable (False) the mixer.

class pipecat.frames.frames.ServiceSwitcherFrame

Bases: ControlFrame

A base class for frames that affect ServiceSwitcher behavior.

class pipecat.frames.frames.ManuallySwitchServiceFrame(service: FrameProcessor)

Bases: ServiceSwitcherFrame

A frame to request a manual switch in the active service in a ServiceSwitcher.

Handled by ServiceSwitcherStrategyManual to switch the active service.

service: FrameProcessor