transport

Tavus transport implementation for Pipecat.

This module provides integration with the Tavus platform for creating conversational AI applications with avatars. It manages conversation sessions and provides real-time audio/video streaming capabilities through the Tavus API.

class pipecat.transports.tavus.transport.TavusApi(api_key: str, session: ClientSession)[source]

Bases: object

Helper class for interacting with the Tavus API (v2).

Provides methods for creating and managing conversations with Tavus avatars, including conversation lifecycle management and persona information retrieval.

BASE_URL = 'https://tavusapi.com/v2'

MOCK_CONVERSATION_ID = 'dev-conversation'

MOCK_PERSONA_NAME = 'TestTavusTransport'

__init__(api_key: str, session: ClientSession)[source]

Initialize the TavusApi client.

Parameters:

api_key – Tavus API key for authentication.
session – An aiohttp session for making HTTP requests.

async create_conversation(replica_id: str, persona_id: str) → dict[source]

Create a new conversation with the specified replica and persona.

Parameters:

replica_id – ID of the replica to use in the conversation.
persona_id – ID of the persona to use in the conversation.

Returns:

Dictionary containing conversation_id and conversation_url.

async end_conversation(conversation_id: str)[source]

End an existing conversation.

Parameters:: conversation_id – ID of the conversation to end.

async get_persona_name(persona_id: str) → str[source]

Get the name of a persona by ID.

Parameters:: persona_id – ID of the persona to retrieve.
Returns:: The name of the persona.

class pipecat.transports.tavus.transport.TavusCallbacks(*, on_joined: Callable[[Mapping[str, Any]], Awaitable[None]], on_participant_joined: Callable[[Mapping[str, Any]], Awaitable[None]], on_participant_left: Callable[[Mapping[str, Any], str], Awaitable[None]])[source]

Bases: BaseModel

Callback handlers for Tavus events.

Parameters:

on_joined – Called when the bot joins the Daily room.
on_participant_joined – Called when a participant joins the conversation.
on_participant_left – Called when a participant leaves the conversation.

on_joined: Callable[[Mapping[str, Any]], Awaitable[None]]

on_participant_joined: Callable[[Mapping[str, Any]], Awaitable[None]]

on_participant_left: Callable[[Mapping[str, Any], str], Awaitable[None]]

class pipecat.transports.tavus.transport.TavusParams(*, audio_out_enabled: bool = True, audio_out_sample_rate: int | None = None, audio_out_channels: int = 1, audio_out_bitrate: int = 96000, audio_out_10ms_chunks: int = 4, audio_out_mixer: Mapping[str | None, ~pipecat.audio.mixers.base_audio_mixer.BaseAudioMixer] | None=None, audio_out_destinations: list[str] = <factory>, audio_out_end_silence_secs: int = 2, audio_out_auto_silence: bool = True, audio_in_enabled: bool = True, audio_in_sample_rate: int | None = None, audio_in_channels: int = 1, audio_in_filter: BaseAudioFilter | None = None, audio_in_stream_on_start: bool = True, audio_in_passthrough: bool = True, video_in_enabled: bool = False, video_out_enabled: bool = False, video_out_is_live: bool = False, video_out_width: int = 1024, video_out_height: int = 768, video_out_bitrate: int | None = None, video_out_framerate: int = 30, video_out_color_format: str = 'RGB', video_out_codec: str | None = None, video_out_destinations: list[str] = <factory>, api_url: str = 'https://api.daily.co/v1', api_key: str = '', audio_in_user_tracks: bool = True, camera_out_enabled: bool = True, camera_out_send_settings: dict[str, ~typing.Any] | None=None, custom_audio_track_params: Mapping[str, ~pipecat.transports.daily.transport.DailyCustomAudioTrackParams] | None=None, custom_video_track_params: Mapping[str, ~pipecat.transports.daily.transport.DailyCustomVideoTrackParams] | None=None, dialin_settings: DailyDialinSettings | None = None, microphone_out_enabled: bool = False, transcription_enabled: bool = False, transcription_settings: DailyTranscriptionSettings = DailyTranscriptionSettings(language='en', model='nova-2-general', profanity_filter=True, redact=False, endpointing=True, punctuate=True, includeRawResponse=True, extra={'interim_results': True}), audio_out_faster_than_realtime: bool = False)[source]

Bases: DailyParams

Configuration parameters for the Tavus transport.

Parameters:

audio_in_enabled – Whether to enable audio input from participants.
audio_out_enabled – Whether to enable audio output to participants.
microphone_out_enabled – Whether to enable microphone output track.
audio_out_faster_than_realtime – Whether to send bot audio app messages as fast as possible instead of paced to real playback time. Speeds up avatar rendering, but breaks any downstream processor (e.g. AudioBufferProcessor) that relies on bot audio arriving at the pipeline at roughly real time.

audio_in_enabled: bool

audio_out_enabled: bool

microphone_out_enabled: bool

audio_out_faster_than_realtime: bool

class pipecat.transports.tavus.transport.TavusTransportClient(*, bot_name: str, params: TavusParams = TavusParams(audio_out_enabled=True, audio_out_sample_rate=None, audio_out_channels=1, audio_out_bitrate=96000, audio_out_10ms_chunks=4, audio_out_mixer=None, audio_out_destinations=[], audio_out_end_silence_secs=2, audio_out_auto_silence=True, audio_in_enabled=True, audio_in_sample_rate=None, audio_in_channels=1, audio_in_filter=None, audio_in_stream_on_start=True, audio_in_passthrough=True, video_in_enabled=False, video_out_enabled=False, video_out_is_live=False, video_out_width=1024, video_out_height=768, video_out_bitrate=None, video_out_framerate=30, video_out_color_format='RGB', video_out_codec=None, video_out_destinations=[], api_url='https://api.daily.co/v1', api_key='', audio_in_user_tracks=True, camera_out_enabled=True, camera_out_send_settings=None, custom_audio_track_params=None, custom_video_track_params=None, dialin_settings=None, microphone_out_enabled=False, transcription_enabled=False, transcription_settings=DailyTranscriptionSettings(language='en', model='nova-2-general', profanity_filter=True, redact=False, endpointing=True, punctuate=True, includeRawResponse=True, extra={'interim_results': True}), audio_out_faster_than_realtime=False), callbacks: TavusCallbacks, api_key: str, replica_id: str, persona_id: str = 'pipecat0', session: ClientSession)[source]

Bases: object

Transport client that integrates Pipecat with the Tavus platform.

A transport client that integrates a Pipecat Bot with the Tavus platform by managing conversation sessions using the Tavus API.

This client uses TavusApi to interact with the Tavus backend services. When a conversation is started via TavusApi, Tavus provides a roomURL that can be used to connect the Pipecat Bot into the same virtual room where the TavusBot is operating.

__init__(*, bot_name: str, params: TavusParams = TavusParams(audio_out_enabled=True, audio_out_sample_rate=None, audio_out_channels=1, audio_out_bitrate=96000, audio_out_10ms_chunks=4, audio_out_mixer=None, audio_out_destinations=[], audio_out_end_silence_secs=2, audio_out_auto_silence=True, audio_in_enabled=True, audio_in_sample_rate=None, audio_in_channels=1, audio_in_filter=None, audio_in_stream_on_start=True, audio_in_passthrough=True, video_in_enabled=False, video_out_enabled=False, video_out_is_live=False, video_out_width=1024, video_out_height=768, video_out_bitrate=None, video_out_framerate=30, video_out_color_format='RGB', video_out_codec=None, video_out_destinations=[], api_url='https://api.daily.co/v1', api_key='', audio_in_user_tracks=True, camera_out_enabled=True, camera_out_send_settings=None, custom_audio_track_params=None, custom_video_track_params=None, dialin_settings=None, microphone_out_enabled=False, transcription_enabled=False, transcription_settings=DailyTranscriptionSettings(language='en', model='nova-2-general', profanity_filter=True, redact=False, endpointing=True, punctuate=True, includeRawResponse=True, extra={'interim_results': True}), audio_out_faster_than_realtime=False), callbacks: TavusCallbacks, api_key: str, replica_id: str, persona_id: str = 'pipecat0', session: ClientSession) → None[source]

Initialize the Tavus transport client.

Parameters:

bot_name – The name of the Pipecat bot instance.
params – Optional parameters for Tavus operation.
callbacks – Callback handlers for Tavus-related events.
api_key – API key for authenticating with Tavus API.
replica_id – ID of the replica to use in the Tavus conversation.
persona_id – ID of the Tavus persona. Defaults to “pipecat0”, which signals Tavus to use the TTS voice of the Pipecat bot instead of a Tavus persona voice, and to expect audio over the conversation.echo app message API.
session – The aiohttp session for making async HTTP requests.

async setup(setup: FrameProcessorSetup)[source]

Setup the client and initialize the conversation.

Parameters:: setup – The frame processor setup configuration.

async cleanup()[source]: Cleanup client resources.

property conversation_id: str | None: Get the current conversation ID.

async get_persona_name() → str[source]

Get the persona name from the API.

Returns:: The name of the current persona.

async start(frame: StartFrame)[source]

Start the client and join the room.

Parameters:: frame – The start frame containing initialization parameters.

async stop()[source]: Stop the client and end the conversation.

async capture_participant_video(participant_id: str, callback: Callable, framerate: int = 30, video_source: str = 'camera', color_format: str = 'RGB')[source]

Capture video from a participant.

Parameters:

participant_id – ID of the participant to capture video from.
callback – Callback function to handle video frames.
framerate – Desired framerate for video capture.
video_source – Video source to capture from.
color_format – Color format for video frames.

async capture_participant_audio(participant_id: str, callback: Callable, audio_source: str = 'microphone', sample_rate: int = 16000, callback_interval_ms: int = 20)[source]

Capture audio from a participant.

Parameters:

participant_id – ID of the participant to capture audio from.
callback – Callback function to handle audio data.
audio_source – Audio source to capture from.
sample_rate – Desired sample rate for audio capture.
callback_interval_ms – Interval between audio callbacks in milliseconds.

async send_message(frame: OutputTransportMessageFrame | OutputTransportMessageUrgentFrame)[source]

Send a message to participants.

Parameters:: frame – The message frame to send.

property out_sample_rate: int

Get the output sample rate.

Returns:: The output sample rate in Hz.

property in_sample_rate: int

Get the input sample rate.

Returns:: The input sample rate in Hz.

async send_interrupt_message() → None[source]: Send an interrupt message to the conversation.

async encode_audio_and_send(audio: bytes, done: bool, inference_id: str | None) → None[source]

Base64-encode audio bytes and send as a conversation.echo app message.

Parameters:

audio – Raw PCM bytes at the client output sample rate, 16-bit mono.
done – True when this is the final chunk for the current inference.
inference_id – Identifier tying all chunks of one utterance together.

async start_send_task() → None[source]: Start the audio accumulation and send task.

async cancel_send_task() → None[source]: Cancel the send task and discard any buffered audio.

async queue_tts_frame(frame: OutputAudioRawFrame | TTSStoppedFrame) → bool[source]

Add an audio frame or end-of-utterance signal to the send queue.

Parameters:: frame – An audio frame, or TTSStoppedFrame signalling end of utterance.
Returns:: True if the frame was queued, False if the queue is not active.

async send_realtime_audio_frame(frame: OutputAudioRawFrame) → bool[source]

Send a single audio frame immediately, paced to real playback time.

Used instead of the queue/send-task path when audio isn’t sent faster than realtime: the caller (TavusOutputTransport) already paces one frame at a time.

Parameters:: frame – The audio frame to send.
Returns:: True, since the message is sent synchronously.

async end_realtime_utterance() → None[source]: Close the current realtime utterance, if any, with a done marker.

reset_realtime_utterance() → None[source]: Discard the current realtime utterance without sending a done marker.

async update_subscriptions(participant_settings=None, profile_settings=None)[source]

Update subscription settings for participants.

Parameters:

participant_settings – Per-participant subscription settings.
profile_settings – Global subscription profile settings.

async register_audio_destination(destination: str, auto_silence: bool | None = True)[source]

Register an audio destination for output.

Parameters:

destination – The destination identifier to register.
auto_silence – If True, the audio source inserts silence when no audio is available. If False, the source waits for audio data. Defaults to True.

class pipecat.transports.tavus.transport.TavusInputTransport(client: TavusTransportClient, params: TransportParams, **kwargs)[source]

Bases: BaseInputTransport

Input transport for receiving audio and events from Tavus conversations.

Handles incoming audio streams from participants and manages audio capture from the Daily room connected to the Tavus conversation.

__init__(client: TavusTransportClient, params: TransportParams, **kwargs)[source]

Initialize the Tavus input transport.

Parameters:

client – The Tavus transport client instance.
params – Transport configuration parameters.
**kwargs – Additional arguments passed to parent class.

async setup(setup: FrameProcessorSetup)[source]

Setup the input transport.

Parameters:: setup – The frame processor setup configuration.

async cleanup()[source]: Cleanup input transport resources.

async start(frame: StartFrame)[source]

Start the input transport.

Parameters:: frame – The start frame containing initialization parameters.

async stop(frame: EndFrame)[source]

Stop the input transport.

Parameters:: frame – The end frame signaling transport shutdown.

async cancel(frame: CancelFrame)[source]

Cancel the input transport.

Parameters:: frame – The cancel frame signaling immediate cancellation.

async start_capturing_audio(participant)[source]

Start capturing audio from a participant.

Parameters:: participant – The participant to capture audio from.

class pipecat.transports.tavus.transport.TavusOutputTransport(client: TavusTransportClient, params: TavusParams, **kwargs)[source]

Bases: BaseOutputTransport

Output transport for sending audio and events to Tavus conversations.

Handles outgoing audio streams to participants and manages the custom audio track expected by the Tavus platform.

__init__(client: TavusTransportClient, params: TavusParams, **kwargs)[source]

Initialize the Tavus output transport.

Parameters:

client – The Tavus transport client instance.
params – Transport configuration parameters.
**kwargs – Additional arguments passed to parent class.

async setup(setup: FrameProcessorSetup)[source]

Setup the output transport.

Parameters:: setup – The frame processor setup configuration.

async cleanup()[source]: Cleanup output transport resources.

async push_frame(frame: Frame, direction: FrameDirection = FrameDirection.DOWNSTREAM)[source]: Signal end of utterance when bot stops speaking.

async start(frame: StartFrame)[source]

Start the output transport.

Parameters:: frame – The start frame containing initialization parameters.

async stop(frame: EndFrame)[source]

Stop the output transport.

Parameters:: frame – The end frame signaling transport shutdown.

async cancel(frame: CancelFrame)[source]

Cancel the output transport.

Parameters:: frame – The cancel frame signaling immediate cancellation.

async send_message(frame: OutputTransportMessageFrame | OutputTransportMessageUrgentFrame)[source]

Send a message to participants.

Parameters:: frame – The message frame to send.

async process_frame(frame: Frame, direction: FrameDirection)[source]

Process frames and handle interruptions.

Parameters:

frame – The frame to process.
direction – The direction of frame flow in the pipeline.

async write_audio_frame(frame: OutputAudioRawFrame) → bool[source]

Send an audio frame via app message.

Parameters:: frame – The audio frame to write.
Returns:: True if the frame was sent (or queued) successfully.

class pipecat.transports.tavus.transport.TavusTransport(bot_name: str, session: ClientSession, api_key: str, replica_id: str, persona_id: str = 'pipecat0', params: TavusParams = TavusParams(audio_out_enabled=True, audio_out_sample_rate=None, audio_out_channels=1, audio_out_bitrate=96000, audio_out_10ms_chunks=4, audio_out_mixer=None, audio_out_destinations=[], audio_out_end_silence_secs=2, audio_out_auto_silence=True, audio_in_enabled=True, audio_in_sample_rate=None, audio_in_channels=1, audio_in_filter=None, audio_in_stream_on_start=True, audio_in_passthrough=True, video_in_enabled=False, video_out_enabled=False, video_out_is_live=False, video_out_width=1024, video_out_height=768, video_out_bitrate=None, video_out_framerate=30, video_out_color_format='RGB', video_out_codec=None, video_out_destinations=[], api_url='https://api.daily.co/v1', api_key='', audio_in_user_tracks=True, camera_out_enabled=True, camera_out_send_settings=None, custom_audio_track_params=None, custom_video_track_params=None, dialin_settings=None, microphone_out_enabled=False, transcription_enabled=False, transcription_settings=DailyTranscriptionSettings(language='en', model='nova-2-general', profanity_filter=True, redact=False, endpointing=True, punctuate=True, includeRawResponse=True, extra={'interim_results': True}), audio_out_faster_than_realtime=False), input_name: str | None = None, output_name: str | None = None)[source]

Bases: BaseTransport

Transport implementation for Tavus video calls.

When used, the Pipecat bot joins the same virtual room as the Tavus Avatar and the user. This is achieved by using TavusTransportClient, which initiates the conversation via TavusApi and obtains a room URL that all participants connect to.

Event handlers available:

on_connected(transport, data): Bot connected to the room
on_client_connected(transport, participant): Participant connected to the session
on_client_disconnected(transport, participant): Participant disconnected from the session

Example:

@transport.event_handler("on_client_connected")
async def on_client_connected(transport, participant):
    ...

__init__(bot_name: str, session: ClientSession, api_key: str, replica_id: str, persona_id: str = 'pipecat0', params: TavusParams = TavusParams(audio_out_enabled=True, audio_out_sample_rate=None, audio_out_channels=1, audio_out_bitrate=96000, audio_out_10ms_chunks=4, audio_out_mixer=None, audio_out_destinations=[], audio_out_end_silence_secs=2, audio_out_auto_silence=True, audio_in_enabled=True, audio_in_sample_rate=None, audio_in_channels=1, audio_in_filter=None, audio_in_stream_on_start=True, audio_in_passthrough=True, video_in_enabled=False, video_out_enabled=False, video_out_is_live=False, video_out_width=1024, video_out_height=768, video_out_bitrate=None, video_out_framerate=30, video_out_color_format='RGB', video_out_codec=None, video_out_destinations=[], api_url='https://api.daily.co/v1', api_key='', audio_in_user_tracks=True, camera_out_enabled=True, camera_out_send_settings=None, custom_audio_track_params=None, custom_video_track_params=None, dialin_settings=None, microphone_out_enabled=False, transcription_enabled=False, transcription_settings=DailyTranscriptionSettings(language='en', model='nova-2-general', profanity_filter=True, redact=False, endpointing=True, punctuate=True, includeRawResponse=True, extra={'interim_results': True}), audio_out_faster_than_realtime=False), input_name: str | None = None, output_name: str | None = None)[source]

Initialize the Tavus transport.

Parameters:

bot_name – The name of the Pipecat bot.
session – aiohttp session used for async HTTP requests.
api_key – Tavus API key for authentication.
replica_id – ID of the replica model used for voice generation.
persona_id – ID of the Tavus persona. Defaults to “pipecat0”, which signals Tavus to use the TTS voice of the Pipecat bot instead of a Tavus persona voice, and to expect audio over the conversation.echo app message API.
params – Optional Tavus-specific configuration parameters.
input_name – Optional name for the input transport.
output_name – Optional name for the output transport.

async update_subscriptions(participant_settings=None, profile_settings=None)[source]

Update subscription settings for participants.

Parameters:

participant_settings – Per-participant subscription settings.
profile_settings – Global subscription profile settings.

input() → FrameProcessor[source]

Get the input transport for receiving media and events.

Returns:: The Tavus input transport instance.

output() → FrameProcessor[source]

Get the output transport for sending media and events.

Returns:: The Tavus output transport instance.