transport

LiveKit transport implementation for Pipecat.

This module provides comprehensive LiveKit real-time communication integration including audio streaming, data messaging, participant management, and room event handling for conversational AI applications.

class pipecat.transports.livekit.transport.LiveKitInputTransportMessageFrame(message: Any, participant_id: str | None = None)[source]

Bases: InputTransportMessageFrame

Frame for incoming transport messages from LiveKit rooms.

Parameters:

participant_id – ID of the participant this message is from.

participant_id: str | None = None
class pipecat.transports.livekit.transport.LiveKitOutputTransportMessageFrame(message: Any, participant_id: str | None = None)[source]

Bases: OutputTransportMessageFrame

Frame for transport messages in LiveKit rooms.

Parameters:

participant_id – Optional ID of the participant this message is for/from.

participant_id: str | None = None
class pipecat.transports.livekit.transport.LiveKitOutputTransportMessageUrgentFrame(message: Any, participant_id: str | None = None)[source]

Bases: OutputTransportMessageUrgentFrame

Frame for urgent transport messages in LiveKit rooms.

Parameters:

participant_id – Optional ID of the participant this message is for/from.

participant_id: str | None = None
class pipecat.transports.livekit.transport.LiveKitParams(*, audio_out_enabled: bool = False, audio_out_sample_rate: int | None = None, audio_out_channels: int = 1, audio_out_bitrate: int = 96000, audio_out_10ms_chunks: int = 4, audio_out_filter: BaseAudioFilter | None = None, audio_out_volume: float = 1.0, audio_out_mixer: Mapping[str | None, ~pipecat.audio.mixers.base_audio_mixer.BaseAudioMixer] | None=None, audio_out_destinations: list[str] = <factory>, audio_out_end_silence_secs: int = 2, audio_out_auto_silence: bool = True, audio_out_write_timeout_secs: float = 10.0, audio_in_enabled: bool = False, audio_in_sample_rate: int | None = None, audio_in_channels: int = 1, audio_in_filter: BaseAudioFilter | None = None, audio_in_stream_on_start: bool = True, audio_in_passthrough: bool = True, video_in_enabled: bool = False, video_in_sources: dict[str, ~pipecat.transports.base_transport.VideoInSourceParams]=<factory>, video_out_enabled: bool = False, video_out_is_live: bool = False, video_out_width: int = 1024, video_out_height: int = 768, video_out_bitrate: int | None = None, video_out_framerate: int = 30, video_out_color_format: str = 'RGB', video_out_codec: str | None = None, video_out_destinations: list[str] = <factory>, audio_in_user_tracks: bool = True, audio_out_queue_size_ms: int = 1000, video_out_max_bitrate: int | None = None)[source]

Bases: TransportParams

Configuration parameters for LiveKit transport.

Video output publishes a single "pipecat-video" camera track (mirroring how audio output always publishes one "pipecat-audio" microphone track) when video_out_enabled is set. The track is sized using video_out_width/video_out_height and encodes frames according to video_out_color_format (default "RGB"); video_out_framerate governs how often BaseOutputTransport draws frames. Per-destination video routing (multiple named output tracks, as supported by Daily’s camera_out_enabled/register_video_destination) is not yet implemented for LiveKit.

video_out_codec selects the published video codec ("VP8", "H264", "VP9", "AV1" or "H265"); LiveKit picks one when it is unset.

With video_in_enabled, a user’s camera ("camera") and screen share ("screenVideo") are received separately. Without video_in_sources, every frame of every video track is passed on. With it, only the listed sources are, each at its own framerate, plus any source captured with LiveKitTransport.capture_participant_video().

Parameters:
  • audio_in_user_tracks – Receive each participant’s audio as its own stream of ``UserAudioRawFrame``s tagged with their identity (the default). All of these streams enter the pipeline through the one input transport, so a pipeline that runs a single STT and VAD should only receive one of them. When False, the participants’ audio is mixed into a single stream of ``InputAudioRawFrame``s.

  • audio_out_queue_size_ms – Buffer size of the outgoing audio source, in milliseconds (LiveKit’s default is 1000).

  • video_out_max_bitrate – Maximum bitrate of the published video track, in bits per second, capped at video_out_framerate. LiveKit chooses the encoding from the track resolution when unset.

class pipecat.transports.livekit.transport.LiveKitCallbacks(*, on_connected: Callable[[], Awaitable[None]], on_disconnected: Callable[[], Awaitable[None]], on_before_disconnect: Callable[[], Awaitable[None]], on_participant_connected: Callable[[str], Awaitable[None]], on_participant_disconnected: Callable[[str], Awaitable[None]], on_audio_track_subscribed: Callable[[str], Awaitable[None]], on_audio_track_unsubscribed: Callable[[str], Awaitable[None]], on_video_track_subscribed: Callable[[str], Awaitable[None]], on_video_track_unsubscribed: Callable[[str, str], Awaitable[None]], on_data_received: Callable[[bytes, str | None], Awaitable[None]], on_first_participant_joined: Callable[[str], Awaitable[None]], on_dtmf_event: Callable[[Any], Awaitable[None]], on_active_speaker_changed: Callable[[str], Awaitable[None]], on_video_track_muted: Callable[[str, str], Awaitable[None]])[source]

Bases: BaseModel

Callback handlers for LiveKit events.

Parameters:
  • on_connected – Called when connected to the LiveKit room.

  • on_disconnected – Called when disconnected from the LiveKit room.

  • on_participant_connected – Called when a participant joins the room.

  • on_participant_disconnected – Called when a participant leaves the room.

  • on_audio_track_subscribed – Called when an audio track is subscribed.

  • on_audio_track_unsubscribed – Called when an audio track is unsubscribed.

  • on_data_received – Called when data is received. The sender is None for packets sent by a server SDK, which LiveKit delivers unattributed.

  • on_first_participant_joined – Called when the first participant joins.

  • on_dtmf_event – Called when a SIP DTMF tone is received.

  • on_active_speaker_changed – Called with the identity of the room’s loudest speaker when it changes. The bot can be the speaker while it talks.

  • on_video_track_muted – Called with the participant and video source when a participant mutes a video track, e.g. turns their camera off.

on_before_disconnect: Callable[[], Awaitable[None]]
on_video_track_subscribed: Callable[[str], Awaitable[None]]
on_video_track_unsubscribed: Callable[[str, str], Awaitable[None]]
class pipecat.transports.livekit.transport.LiveKitTransportClient(url: str, token: str, room_name: str, params: LiveKitParams, callbacks: LiveKitCallbacks, transport_name: str)[source]

Bases: object

Core client for interacting with LiveKit rooms.

Manages the connection to LiveKit rooms and handles all low-level API interactions including room management, audio streaming, data messaging, and event handling.

__init__(url: str, token: str, room_name: str, params: LiveKitParams, callbacks: LiveKitCallbacks, transport_name: str)[source]

Initialize the LiveKit transport client.

Parameters:
  • url – LiveKit server URL to connect to.

  • token – Authentication token for the room.

  • room_name – Name of the LiveKit room to join.

  • params – Configuration parameters for the transport.

  • callbacks – Event callback handlers.

  • transport_name – Name identifier for the transport.

property participant_id: str

Get the participant ID for this client.

Returns:

The participant ID assigned by LiveKit.

property room: Room

Get the LiveKit room instance.

Returns:

The LiveKit room object.

Raises:

Exception – If room object is not available.

async setup(setup: FrameProcessorSetup)[source]

Setup the client with task manager and room initialization.

Parameters:

setup – The frame processor setup configuration.

async cleanup()[source]

Cleanup client resources.

async connect()[source]

Connect to the LiveKit room with retry logic.

async disconnect()[source]

Disconnect from the LiveKit room.

async send_data(data: bytes, participant_id: str | None = None)[source]

Send data to participants in the room.

Parameters:
  • data – The data bytes to send.

  • participant_id – Optional specific participant to send to.

async send_dtmf(digit: str)[source]

Send DTMF tone to the room.

Parameters:

digit – The DTMF digit to send (0-9, *, #).

async publish_audio(audio_frame: AudioFrame) → bool[source]

Publish an audio frame to the room.

Parameters:

audio_frame – The LiveKit audio frame to publish.

async publish_video(video_frame: VideoFrame) → bool[source]

Publish a video frame to the room.

Parameters:

video_frame – The LiveKit video frame to publish.

Returns:

True if the video frame was published successfully, False otherwise.

get_participants() → list[str][source]

Get list of participant IDs in the room.

Returns:

List of participant LiveKit identities.

async get_participant_metadata(participant_id: str) → dict[source]

Get metadata for a specific participant.

Parameters:

participant_id – LiveKit identity of the participant to get metadata for.

Returns:

Dictionary containing participant metadata.

async set_participant_metadata(metadata: str)[source]

Set metadata for the local participant.

Parameters:

metadata – Metadata string to set.

async mute_participant(participant_id: str)[source]

Stop receiving a specific participant’s audio.

LiveKit doesn’t let one participant force-mute another’s microphone; this unsubscribes the bot from their audio track instead.

Parameters:

participant_id – LiveKit identity of the participant to stop receiving audio from.

async unmute_participant(participant_id: str)[source]

Resume receiving a specific participant’s audio.

Parameters:

participant_id – LiveKit identity of the participant to resume receiving audio from.

async get_next_audio_frame()[source]

Get the next audio frame from the queue.

video_source_enabled(participant_id: str, video_source: str) → bool[source]

Whether a participant is sending a video source.

A participant who turns their camera off usually mutes the track rather than unpublishing it, so a subscribed track can still send nothing.

Parameters:
  • participant_id – The participant’s identity.

  • video_source – The video source, "camera" or "screenVideo".

Returns:

Whether the source’s track is subscribed and not muted.

async get_next_video_frame()[source]

Get the next video frame from the queue.

class pipecat.transports.livekit.transport.LiveKitInputTransport(transport: BaseTransport, client: LiveKitTransportClient, params: LiveKitParams, **kwargs)[source]

Bases: BaseInputTransport

Handles incoming media streams and events from LiveKit rooms.

Processes incoming audio streams from room participants and forwards them as Pipecat frames, including audio resampling and VAD integration.

__init__(transport: BaseTransport, client: LiveKitTransportClient, params: LiveKitParams, **kwargs)[source]

Initialize the LiveKit input transport.

Parameters:
  • transport – The parent transport instance.

  • client – LiveKitTransportClient instance.

  • params – Configuration parameters.

  • **kwargs – Additional arguments passed to parent class.

async setup(setup: FrameProcessorSetup)[source]

Setup the input transport with shared client setup.

Parameters:

setup – The frame processor setup configuration.

async cleanup()[source]

Release input transport resources at teardown.

async start(frame: StartFrame)[source]

Start receiving media from the LiveKit room.

Parameters:

frame – The start frame containing initialization parameters.

async stop(frame: EndFrame)[source]

Stop the input transport and disconnect from LiveKit room.

Parameters:

frame – The end frame signaling transport shutdown.

async cancel(frame: CancelFrame)[source]

Cancel the input transport and disconnect from LiveKit room.

Parameters:

frame – The cancel frame signaling immediate cancellation.

async push_app_message(message: Any, sender: str | None)[source]

Push an application message as an urgent transport frame.

Broadcast both upstream and downstream so it reaches RTVIProcessor regardless of where it sits relative to the transport in the pipeline.

Parameters:
  • message – The message data to send.

  • sender – ID of the message sender, or None if it was sent unattributed by a server SDK.

release_participant(participant_id: str)[source]

Drop a participant’s resampler once their audio track is gone.

async process_frame(frame: Frame, direction: FrameDirection)[source]

Process incoming frames, answering user image requests.

Parameters:
  • frame – The frame to process.

  • direction – The direction of frame flow in the pipeline.

async capture_participant_video(participant_id: str, framerate: int | None = 30, video_source: str = 'camera', *, on_request_only: bool = False)[source]

Capture a participant’s video source at a framerate.

This takes precedence over video_in_sources for the source.

Parameters:
  • participant_id – The participant’s identity.

  • framerate – Frames per second to pass on, or None for every frame. It doesn’t apply with on_request_only.

  • video_source – The video source, "camera" or "screenVideo".

  • on_request_only – Pass on only the frames that answer image requests.

async request_participant_image(frame: UserImageRequestFrame)[source]

Request a video frame from a specific participant.

Parameters:

frame – The user image request frame.

async stop_participant_video(participant_id: str, video_source: str)[source]

Answer the image requests waiting on a video source that stopped.

The source stays captured, so its frames are sampled again if it restarts.

Parameters:
  • participant_id – The participant’s identity.

  • video_source – The video source that stopped.

async remove_participant_video(participant_id: str)[source]

Stop sampling video from a participant who left.

Parameters:

participant_id – The participant’s identity.

async remove_all_video()[source]

Stop sampling video from every participant, e.g. after leaving the room.

class pipecat.transports.livekit.transport.LiveKitOutputTransport(transport: BaseTransport, client: LiveKitTransportClient, params: LiveKitParams, **kwargs)[source]

Bases: BaseOutputTransport

Handles outgoing media streams and events to LiveKit rooms.

Manages sending audio and video frames and data messages to LiveKit room participants, including audio/video format conversion for LiveKit compatibility. Video output publishes to a single default camera track when LiveKitParams.video_out_enabled is set.

__init__(transport: BaseTransport, client: LiveKitTransportClient, params: LiveKitParams, **kwargs)[source]

Initialize the LiveKit output transport.

Parameters:
  • transport – The parent transport instance.

  • client – LiveKitTransportClient instance.

  • params – Configuration parameters.

  • **kwargs – Additional arguments passed to parent class.

async setup(setup: FrameProcessorSetup)[source]

Setup the output transport with shared client setup.

Parameters:

setup – The frame processor setup configuration.

async cleanup()[source]

Release output transport resources at teardown.

async start(frame: StartFrame)[source]

Start the output transport.

Parameters:

frame – The start frame containing initialization parameters.

async stop(frame: EndFrame)[source]

Stop the output transport and disconnect from LiveKit room.

Parameters:

frame – The end frame signaling transport shutdown.

async cancel(frame: CancelFrame)[source]

Cancel the output transport and disconnect from LiveKit room.

Parameters:

frame – The cancel frame signaling immediate cancellation.

async process_frame(frame: Frame, direction: FrameDirection)[source]

Process frames, clearing the LiveKit AudioSource buffer on interruption.

When an InterruptionFrame arrives, any audio already submitted to the LiveKit AudioSource (but not yet played out) is cleared immediately so the bot stops speaking without delay.

Parameters:
  • frame – The frame to process.

  • direction – The direction of frame flow in the pipeline.

async send_message(frame: OutputTransportMessageFrame | OutputTransportMessageUrgentFrame)[source]

Send a transport message to participants.

Parameters:

frame – The transport message frame to send.

async write_audio_frame(frame: OutputAudioRawFrame) → bool[source]

Write an audio frame to the LiveKit room.

Parameters:

frame – The audio frame to write.

Returns:

True if the audio frame was written successfully, False otherwise.

async write_video_frame(frame: OutputImageRawFrame) → bool[source]

Write a video frame to the LiveKit room’s published camera track.

Publishes to the single default video track set up in LiveKitTransportClient.connect (mirroring how audio always publishes to one microphone track). Per-destination routing to multiple named video tracks is not supported yet, so frame.transport_destination is ignored.

Parameters:

frame – The video frame to write.

Returns:

True if the video frame was written successfully, False otherwise.

class pipecat.transports.livekit.transport.LiveKitTransport(url: str, token: str, room_name: str, params: LiveKitParams | None = None, input_name: str | None = None, output_name: str | None = None)[source]

Bases: BaseTransport

Transport implementation for LiveKit real-time communication.

Provides comprehensive LiveKit integration including audio streaming, data messaging, participant management, and room event handling for conversational AI applications.

Every participant_id surfaced by this transport (event args, frame fields, get_participants(), and the get_participant_metadata/ mute_participant/unmute_participant methods) is the participant’s LiveKit identity (rtc.Participant.identity) — the value set when minting its access token, and what LiveKit itself keys room.remote_participants by and expects in destination_identities. It is not the participant’s SID (rtc.Participant.sid), a per-connection session id that changes on every reconnect.

Event handlers available:

  • on_connected: Called when the bot connects to the room.

  • on_disconnected: Called when the bot disconnects from the room.

  • on_before_disconnect: [sync] Called just before the bot disconnects.

  • on_call_state_updated: Called when the call state changes. Args: (state: str)

  • on_first_participant_joined: Called when the first participant joins. Args: (participant_id: str)

  • on_participant_connected: Called when a participant connects. Args: (participant_id: str)

  • on_participant_disconnected: Called when a participant disconnects. Args: (participant_id: str)

  • on_participant_left: Called when a participant leaves. Args: (participant_id: str, reason: str)

  • on_client_connected: Called when a participant connects (alias for on_participant_connected). Args: (participant: dict)

  • on_client_disconnected: Called when a participant disconnects (alias for on_participant_disconnected). Args: (participant: dict)

  • on_audio_track_subscribed: Called when an audio track is subscribed. Args: (participant_id: str)

  • on_audio_track_unsubscribed: Called when an audio track is unsubscribed. Args: (participant_id: str)

  • on_video_track_subscribed: Called when a video track is subscribed. Args: (participant_id: str)

  • on_video_track_unsubscribed: Called when a video track is unsubscribed. Args: (participant_id: str)

  • on_app_message: Called when data is received from a participant. RTVI-compatible version of on_data_received. Args: (message: Any, sender: str)

  • on_data_received: Called when data is received. The participant ID is None for packets sent by a server SDK, which LiveKit delivers unattributed. Args: (data: bytes, participant_id: str | None)

  • on_dtmf_event: Called when a SIP DTMF tone is received from a participant. Args: (data: dict) with keys tone/digit, code, and participant_id. Also pushes an InputDTMFFrame so DTMFAggregator works on LiveKit SIP calls.

  • on_active_speaker_changed: Called when the room’s loudest speaker changes, as on Daily. The bot can be the speaker while it talks. Args: (participant: dict) with key id, the speaker’s identity.

Example:

@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant_id):
    await task.queue_frame(TTSSpeakFrame("Hello!"))

@transport.event_handler("on_participant_disconnected")
async def on_participant_disconnected(transport, participant_id):
    await task.queue_frame(EndFrame())
__init__(url: str, token: str, room_name: str, params: LiveKitParams | None = None, input_name: str | None = None, output_name: str | None = None)[source]

Initialize the LiveKit transport.

Parameters:
  • url – LiveKit server URL to connect to.

  • token – Authentication token for the room.

  • room_name – Name of the LiveKit room to join.

  • params – Configuration parameters for the transport.

  • input_name – Optional name for the input transport.

  • output_name – Optional name for the output transport.

get_client_id(client: Any) → str[source]

The id of a client, as passed to on_client_connected.

Parameters:

client – The client, as passed to the transport’s client events.

Returns:

The participant’s identity.

input() → LiveKitInputTransport[source]

Get the input transport for receiving media and events.

Returns:

The LiveKit input transport instance.

output() → LiveKitOutputTransport[source]

Get the output transport for sending media and events.

Returns:

The LiveKit output transport instance.

property participant_id: str

Get the participant ID for this transport.

Returns:

The participant ID assigned by LiveKit.

async send_audio(frame: OutputAudioRawFrame)[source]

Send an audio frame to the LiveKit room.

Parameters:

frame – The audio frame to send.

get_participants() → list[str][source]

Get list of participant IDs in the room.

Returns:

List of participant LiveKit identities.

async get_participant_metadata(participant_id: str) → dict[source]

Get metadata for a specific participant.

Parameters:

participant_id – LiveKit identity of the participant to get metadata for.

Returns:

Dictionary containing participant metadata.

async set_metadata(metadata: str)[source]

Set metadata for the local participant.

Parameters:

metadata – Metadata string to set.

async capture_participant_video(participant_id: str, framerate: int | None = 30, video_source: str = 'camera', *, on_request_only: bool = False)[source]

Capture a participant’s video source at a framerate.

This takes precedence over video_in_sources for the source.

Parameters:
  • participant_id – The participant’s identity.

  • framerate – Frames per second to pass on, or None for every frame. It doesn’t apply with on_request_only.

  • video_source – The video source, "camera" or "screenVideo".

  • on_request_only – Pass on only the frames that answer image requests.

async mute_participant(participant_id: str)[source]

Stop receiving a specific participant’s audio.

Parameters:

participant_id – LiveKit identity of the participant to stop receiving audio from.

async unmute_participant(participant_id: str)[source]

Resume receiving a specific participant’s audio.

Parameters:

participant_id – LiveKit identity of the participant to resume receiving audio from.

async send_message(message: str, participant_id: str | None = None)[source]

Send a message to participants in the room.

Sends at once, ahead of audio still queued in the pipeline. To send the message after queued audio has played, push a LiveKitOutputTransportMessageFrame instead.

Parameters:
  • message – The message string to send.

  • participant_id – Optional specific participant to send to.

async send_message_urgent(message: str, participant_id: str | None = None)[source]

Send an urgent message to participants in the room.

Parameters:
  • message – The urgent message string to send.

  • participant_id – Optional specific participant to send to.

async on_room_event(event)[source]

Handle room events.

Parameters:

event – The room event to handle.

async on_participant_event(event)[source]

Handle participant events.

Parameters:

event – The participant event to handle.

async on_track_event(event)[source]

Handle track events.

Parameters:

event – The track event to handle.