llm_with_backend

An LLM with a backend: a conversational frontend delegating the rest.

The frontend is any LLM service, text or speech-to-speech, and holds the conversation. The backend is a BackendLLMWorker running a heavier model with the tools, and does the work the frontend hands off. LLMWithBackend wraps the frontend so the pair drops into a pipeline where an LLM goes, and installs the tools that join them.

The two exchange messages. The frontend’s delegate tool puts a message to the backend and returns at once; what the backend has to say comes back as a stream of outputs, each appended to the frontend’s conversation as a message marked Backend:, spoken or silent as the backend’s flag says. A second request joins the first, and a correction changes or stops it. How a request is worded is the BackendRequestStrategy’s business, and the BackendConnector owns the tools, the session with the backend, and how each output is rendered into the conversation.

The classes an app builds with import from this package. The rest, such as the prompt texts an app may start from to write its own, import from the module that defines them:

class pipecat.pipeline.llm_with_backend.BackendConnector(*, request_strategy: BackendRequestStrategy | None = None, frontend_pairing_instruction: str = 'THE BACKEND: You have a backend for anything that needs a backend tool or careful reasoning. It sees nothing of the conversation but what you delegate, and nothing you say reaches it: to start, change or stop its work, call the delegate tool, at once, in the same reply, and do the rest of the reply yourself (if any is needed) around it. That holds for every message that concerns its work, whether a new request, a yes to something that was offered, a correction or a "never mind", and even while earlier work is still running: a stop you only speak leaves the work running. Do not delegate what you can do yourself: small talk, a question that a result you already have answers, or a question about how the work is going, which you answer from the backend\'s latest notes or, with none yet, by saying there is no update yet. Never guess a result while waiting. {strategy}\n\nONE ASSISTANT: To the user there is one assistant, you: never mention the backend, delegation, a handoff or tools. Say what is being done as your own doing, such as "I\'m checking", never "the backend is checking", and never that you passed anything along.\n\nAFTER DELEGATING: Acknowledge in one short sentence of your own that fits the request, naming the task briefly if that reads naturally. For example: "Let me look into that", "Sure, I\'ll get on that now" or "Okay, pulling that up now". If you already acknowledged alongside the delegate call, even in part, the user has heard it, and this reply should *not* acknowledge again (though it can "finish the thought" of the previous acknowledgement, if appropriate). Saying it in other words is repeating yourself, and so is adding that you are on it now, that it will take a moment, or that you will report back when it is done; the initial acknowledgement already said all of that. After "Let me look that up for you!", e.g., there is nothing left to say, so say nothing. Finish the thought only when the user asked for something else that you have not answered yet. An acknowledgement is never a substitute for the delegate call. In the acknowledgement, avoid restating the request in full, or describing in detail what you are going to do, or following up on your own with promises, offers or questions: you\'ve fired off the delegate call, so let it be. Then answer whatever else the user asked that you can. Never say work is done, or give a result, that the backend has not given you; if the user asks whether work is under way and you never delegated it, say so and delegate it then.\n\nBACKEND MESSAGES: The backend works on its own after you delegate and may be doing several things at once. What it has to say arrives as messages in the conversation marked "Backend:": a result, a question for the user, or news worth an update, on whatever turn happens to be in progress by then. Such a message is owed to the user, whatever the conversation has moved on to: small talk, other questions or a long wait do not make it unwanted. If the user has said something you have not answered yet, answer that first, then relay the message at the end of that same reply, in your own words. If you have already answered everything the user said, relay just the message; do not repeat or rephrase your earlier reply. Relay each message once. Only when the user has cancelled or changed what they asked for does a message go unsaid, and then say only what still helps. Messages marked "Backend (working):", "Backend (note):" and "Backend (thinking):" are what the backend is doing and thinking. They carry no results: what a call found reaches you only in a message marked "Backend:", so never describe or guess what a call found. Never relay them on their own, but when the user asks how the work is going, answer from the latest of them, concretely: name the step, such as which file it is reading or that it is running the tests, rather than saying only that it is still working. Never state a result you have not received from the backend, and never say work is done that the backend has not said is done.\n\nWhatever else a reply does, a user message about work you delegated, whether to add to it, change it or stop it, a never mind included, gets a delegate call in that reply: nothing else stops or changes that work.', respond_on_delegate: bool = True, timeout_secs: float | None = 30, client_trace: bool = False)[source]

Bases: object

Joins the two layers: the tools the frontend calls, the session with the backend, and what each output becomes.

A request strategy defines the delegate tool and composes the request. It may be given, or left to the connector to pick by frontend kind once bound: TranscriptBackendRequestStrategy for a text frontend, ExplicitBackendRequestStrategy for a speech-to-speech one, whose context lags the audio. A given strategy that reads the frontend’s context (BackendRequestStrategy.requires_context) is refused for a speech-to-speech frontend.

The connector holds the session with the backend for the frontend’s life, and turns each output the backend sends into a message appended to the frontend’s conversation with render_output(), run or not as the output’s prefers_spoken flag says. The function calls the backend makes on the way are reported in the frontend’s pipeline for clients to show; nothing else in the pipeline sees them.

Example:

connector = BackendConnector(respond_on_delegate=False)
__init__(*, request_strategy: BackendRequestStrategy | None = None, frontend_pairing_instruction: str = 'THE BACKEND: You have a backend for anything that needs a backend tool or careful reasoning. It sees nothing of the conversation but what you delegate, and nothing you say reaches it: to start, change or stop its work, call the delegate tool, at once, in the same reply, and do the rest of the reply yourself (if any is needed) around it. That holds for every message that concerns its work, whether a new request, a yes to something that was offered, a correction or a "never mind", and even while earlier work is still running: a stop you only speak leaves the work running. Do not delegate what you can do yourself: small talk, a question that a result you already have answers, or a question about how the work is going, which you answer from the backend\'s latest notes or, with none yet, by saying there is no update yet. Never guess a result while waiting. {strategy}\n\nONE ASSISTANT: To the user there is one assistant, you: never mention the backend, delegation, a handoff or tools. Say what is being done as your own doing, such as "I\'m checking", never "the backend is checking", and never that you passed anything along.\n\nAFTER DELEGATING: Acknowledge in one short sentence of your own that fits the request, naming the task briefly if that reads naturally. For example: "Let me look into that", "Sure, I\'ll get on that now" or "Okay, pulling that up now". If you already acknowledged alongside the delegate call, even in part, the user has heard it, and this reply should *not* acknowledge again (though it can "finish the thought" of the previous acknowledgement, if appropriate). Saying it in other words is repeating yourself, and so is adding that you are on it now, that it will take a moment, or that you will report back when it is done; the initial acknowledgement already said all of that. After "Let me look that up for you!", e.g., there is nothing left to say, so say nothing. Finish the thought only when the user asked for something else that you have not answered yet. An acknowledgement is never a substitute for the delegate call. In the acknowledgement, avoid restating the request in full, or describing in detail what you are going to do, or following up on your own with promises, offers or questions: you\'ve fired off the delegate call, so let it be. Then answer whatever else the user asked that you can. Never say work is done, or give a result, that the backend has not given you; if the user asks whether work is under way and you never delegated it, say so and delegate it then.\n\nBACKEND MESSAGES: The backend works on its own after you delegate and may be doing several things at once. What it has to say arrives as messages in the conversation marked "Backend:": a result, a question for the user, or news worth an update, on whatever turn happens to be in progress by then. Such a message is owed to the user, whatever the conversation has moved on to: small talk, other questions or a long wait do not make it unwanted. If the user has said something you have not answered yet, answer that first, then relay the message at the end of that same reply, in your own words. If you have already answered everything the user said, relay just the message; do not repeat or rephrase your earlier reply. Relay each message once. Only when the user has cancelled or changed what they asked for does a message go unsaid, and then say only what still helps. Messages marked "Backend (working):", "Backend (note):" and "Backend (thinking):" are what the backend is doing and thinking. They carry no results: what a call found reaches you only in a message marked "Backend:", so never describe or guess what a call found. Never relay them on their own, but when the user asks how the work is going, answer from the latest of them, concretely: name the step, such as which file it is reading or that it is running the tests, rather than saying only that it is still working. Never state a result you have not received from the backend, and never say work is done that the backend has not said is done.\n\nWhatever else a reply does, a user message about work you delegated, whether to add to it, change it or stop it, a never mind included, gets a delegate call in that reply: nothing else stops or changes that work.', respond_on_delegate: bool = True, timeout_secs: float | None = 30, client_trace: bool = False)[source]

Initialize the connector.

Parameters:
  • request_strategy – How a delegate call becomes the backend’s request. Picked by frontend kind when omitted.

  • frontend_pairing_instruction – How the model works with a backend, appended to its system instruction: FRONTEND_PAIRING_INSTRUCTION unless given (both constants named here are in prompts). One of its own has to say what that one says: when and how to delegate, that to the user there is one assistant, what to do after delegating, and what to do with the backend’s messages. The request strategy’s guidance goes where PAIRING_INSTRUCTION_STRATEGY_PLACEHOLDER stands, or after the text when it has none.

  • respond_on_delegate – Whether the frontend runs again as soon as a delegation is sent, which is where its acknowledgement comes from: a text model’s tool-call turn carries no prose. Off for a frontend that announces a handoff itself.

  • timeout_secs – How long the backend may take to acknowledge a message, including the wait for it to become ready.

  • client_trace – Whether to send every exchange with the backend to the client as an RTVI server message (type "llm-with-backend"): each request, output, tool-call phase, error and idle. For debugging; a client’s event log shows them.

property request_strategy: BackendRequestStrategy

The request strategy in use. Available once bound.

property tools: list[FunctionSchema]

The tools to install on the frontend. Available once bound.

property tool: FunctionSchema

The delegate tool. Available once bound.

property frontend_instruction: str | None

the pairing instruction with the strategy’s guidance in place. Available once bound.

Type:

What is appended to the frontend’s system instruction

property session: _BackendSession | None

The session with the backend, while one is open.

bind(context: ConnectorContext) → None[source]

Settle the strategy for the layers being joined and build the tools.

Parameters:

context – The layers.

Raises:

ValueError – If the frontend is speech-to-speech and the strategy reads the frontend’s context, which lags the audio there.

build_tools() → list[FunctionSchema][source]

Build the tools to install on the frontend: delegate.

Override to install tools of another shape entirely; the service reads nothing else from the connector but frontend_instruction.

Returns:

The tools, each carrying its handler.

async run_session(worker: BaseWorker, frontend: LLMService[Any]) → None[source]

Hold the session with the backend, delivering what it produces, until cancelled.

Parameters:
  • worker – The pipeline worker the frontend runs in.

  • frontend – The frontend service, which takes the deliveries.

async delegate(params: FunctionCallParams) → None[source]

Put the request a delegate call carries to the backend, and settle the call at once.

Parameters:

params – The delegate call.

async deliver(frontend: LLMService[Any], event: BackendOutput | BackendToolCall | BackendIdle | BackendError) → None[source]

Deliver one thing the backend produced to the frontend.

An output becomes a message appended to the frontend’s conversation, which runs the frontend when the output asks to be spoken. A function call phase is reported as the ExternalFunctionCall*Frame for it, which the RTVI observer turns into function-call events. An error becomes a spoken message, so the user hears the work stopped. The appended messages are uninterruptible: one queued while the bot speaks waits behind that speech, and an interruption clears the speech but must leave the message to be recorded.

Parameters:
  • frontend – The frontend service.

  • event – What the backend produced.

render_output(output: BackendOutput) → Any | LLMSpecificMessage | None[source]

Render a backend output as the message the frontend’s conversation takes in.

Override for another wording, role, or to leave some outputs out by returning None. The default marks the message so the frontend can tell it from the user’s and its own, and so the transcript request strategy can leave it out of what it sends the backend: a spoken output as Backend: with BACKEND_MESSAGE_RELAY_INSTRUCTION after it, a silent one as Backend (note):, a reasoning summary as Backend (thinking):.

Parameters:

output – The output.

Returns:

The message, or None to deliver nothing for this output.

render_tool_call(call: BackendToolCall) → Any | LLMSpecificMessage | None[source]

Render a phase of a backend function call as a message the frontend’s conversation takes in, silently.

The default records each call as it starts, with its arguments, so the frontend can say what the backend is doing if asked; the other phases are not recorded. Override to record more, less, or nothing.

Parameters:

call – The phase of the call.

Returns:

The message, or None to record nothing for this phase.

class pipecat.pipeline.llm_with_backend.BackendRequestStrategy[source]

Bases: object

Defines the delegate tool the frontend calls, and turns a call into the request the backend receives.

The tool’s parameters are this strategy’s to declare, so the connector’s handler has one signature and reads whatever was declared back from params.arguments in compose_request().

tool_parameters: dict[str, Any] = {}

JSON-schema properties of the delegate tool; empty when it takes none.

tool_required: list[str] = []

Which of the properties are required.

frontend_instruction: str | None = None

how the request is worded. It goes where the pairing instruction’s PAIRING_INSTRUCTION_STRATEGY_PLACEHOLDER stands.

Type:

The strategy’s guidance to the frontend model, if any

tool_description: str = ''

How the delegate tool is described to the frontend model.

requires_context: bool = False

Whether the strategy reads the conversation from the frontend’s context, so it needs that context to be current when the delegate tool runs. A speech-to-speech frontend’s context lags the audio, so the connector refuses such a strategy there.

delegate_needs_arguments: bool = True

Whether a delegate call has to carry arguments for the request to be built. Predictive delegation makes its delegate call with none, so it works only with a strategy that sets this False.

async compose_request(params: FunctionCallParams) → str | None[source]

Turn a delegate call into the text put to the backend.

Parameters:

params – The call, carrying the arguments the tool declared and the frontend’s context.

Returns:

The request text, or None when the call carries nothing new for the backend, in which case nothing is sent.

class pipecat.pipeline.llm_with_backend.ExplicitBackendRequestStrategy(*, frontend_instruction: str | None = "Word the request so it stands on its own: the user's goal, the exact details they gave and their latest correction, with everything new they asked for in the one request, so you delegate once per reply however many things that is. Work already handed over stays with the backend: a new request joins it, so do not restate earlier requests; a correction names what changes, and a stop names what to stop.", tool_description: str = "The only way to start, change or stop the backend's work. Hands a request to the backend, which does everything in it that needs a backend tool or careful reasoning, and stops or changes what the user has called off or changed. Call it as soon as any part of what the user says concerns that work, even while the backend is still working, with the request worded to stand on its own, and do the rest yourself. Not for a question about how the work is going, which you answer from the backend's notes. One call per reply, however many things the user asked for. Work already handed over stays with the backend, so send only what is new, including anything the user says about that work, such as to stop or change it. Keep talking with the user while it works.", request_instruction: str = 'If this request asks for something, tell the user the result as soon as you have it, before going on with other work, in a message that begins with >>. If it only stops or changes work already under way, tell them nothing: the assistant already has.')[source]

Bases: BackendRequestStrategy

Has the frontend model word the request itself.

The tool takes a request argument, which is sent to the backend as it stands. The default for a speech-to-speech frontend, which responds to audio before the transcript of that audio reaches its context, so the turns that prompted a handoff may not be recorded when the tool runs.

tool_parameters: dict[str, Any] = {'request': {'description': 'What the user is asking for now, self-contained: their goal, the details they gave (places, dates, names) and their latest correction, or what they said about work already under way, such as to stop it or change it. Work already handed over stays with the backend; a new request joins it, so do not restate earlier requests.', 'type': 'string'}}

JSON-schema properties of the delegate tool; empty when it takes none.

tool_required: list[str] = ['request']

Which of the properties are required.

__init__(*, frontend_instruction: str | None = "Word the request so it stands on its own: the user's goal, the exact details they gave and their latest correction, with everything new they asked for in the one request, so you delegate once per reply however many things that is. Work already handed over stays with the backend: a new request joins it, so do not restate earlier requests; a correction names what changes, and a stop names what to stop.", tool_description: str = "The only way to start, change or stop the backend's work. Hands a request to the backend, which does everything in it that needs a backend tool or careful reasoning, and stops or changes what the user has called off or changed. Call it as soon as any part of what the user says concerns that work, even while the backend is still working, with the request worded to stand on its own, and do the rest yourself. Not for a question about how the work is going, which you answer from the backend's notes. One call per reply, however many things the user asked for. Work already handed over stays with the backend, so send only what is new, including anything the user says about that work, such as to stop or change it. Keep talking with the user while it works.", request_instruction: str = 'If this request asks for something, tell the user the result as soon as you have it, before going on with other work, in a message that begins with >>. If it only stops or changes work already under way, tell them nothing: the assistant already has.')[source]

Initialize the strategy.

Parameters:
  • frontend_instruction – The strategy’s guidance to the frontend model, placed in the pairing instruction: how to word the request.

  • tool_description – How the delegate tool is described.

  • request_instruction – What the backend should do with the request, placed after it.

async compose_request(params: FunctionCallParams) → str | None[source]

Send the model’s request as it stands, with the reporting expectation after it.

class pipecat.pipeline.llm_with_backend.LLMWithBackend(*, frontend: LLMService[Any], backend: BackendLLMWorker | str, connector: BackendConnector | None = None, predictive_delegation: PredictiveDelegation | None = None)[source]

Bases: Pipeline

A conversational LLM, kept fast and light, with a backend that does the work it hands off.

Alpha. The API is unstable: names, arguments, the strategies and the job contract are likely to change between releases.

Put it where the LLM goes in a pipeline. It is a Pipeline, not an LLMService: functions, settings updates, event handlers and a FlowManager go on frontend. It wraps the frontend service, installs the connector’s tools on it as built-in tools, appends the connector’s guidance to the frontend’s system instruction, adds a local backend worker to the pipeline worker so the app never wires it up, and holds the session with the backend for the pipeline’s life. A built-in tool is sent on every inference beside whatever tools the frontend has, and never enters the context’s tool set, so a tool change announced to the model never mentions it. The tools are the backend’s; the frontend has delegate and no more, though a tool it must keep, in its context or configured on the service, stays.

The guidance is the connector’s pairing instruction (how to work with a backend: when and how to delegate, and what to do with the backend’s messages) with the strategy’s (how the request is worded) in place. The frontend’s own system instruction is the place to say what the backend is for, in plain words, as the backend’s own instruction does: no tool names, nothing to update when a tool changes.

Example:

llm = LLMWithBackend(
    frontend=OpenAILLMService(...),
    backend=BackendLLMWorker(
        llm=AnthropicLLMService(...),
        context=LLMContext(tools=[get_weather]),
    ),
)
pipeline = Pipeline([transport.input(), stt, user_agg, llm, tts, transport.output(), assistant_agg])
__init__(*, frontend: LLMService[Any], backend: BackendLLMWorker | str, connector: BackendConnector | None = None, predictive_delegation: PredictiveDelegation | None = None)[source]

Initialize the service.

Parameters:
  • frontend – The conversational LLM service, text or speech-to-speech.

  • backend – The backend worker, or the name of one registered elsewhere (in the app, or in another process on a shared bus). A worker given here is added to the pipeline worker at setup.

  • connector – How delegation works. A default BackendConnector when omitted.

  • predictive_delegation – A PredictiveDelegation, to speed up responses that need backend involvement: its classifier determines whether to wait for the frontend’s inference or go ahead and trigger an early backend delegation. Only for a text frontend.

Raises:

ValueError – If the frontend service declines the role; if the connector’s request strategy reads the frontend’s context and the frontend is speech-to-speech; or if predictive delegation is given with a speech-to-speech frontend or with a request strategy whose delegate call needs arguments.

property frontend: LLMService[Any]

The frontend LLM service.

property backend: BackendLLMWorker | str

The backend worker, or its name.

property connector: BackendConnector

The connector joining the two layers.

property predictive_delegation: PredictiveDelegation | None

The PredictiveDelegation in front of the frontend, if there is one.

property tool: FunctionSchema

The delegate tool installed on the frontend.

async setup(setup: FrameProcessorSetup)[source]

Register a local backend worker as a child of the pipeline worker, set up the frontend, and open the session.

async cleanup()[source]

Close the session with the backend, then clean up the frontend.

class pipecat.pipeline.llm_with_backend.PredictiveDelegation(classifier: BaseClassifier, **kwargs)[source]

Bases: FrameProcessor

An optimization that can speed up responses that need backend involvement.

Alpha. It uses a fast yes/no classifier to determine, after each user request, whether it needs to wait for the frontend’s inference or can go ahead and trigger an early backend delegation. See pipecat.pipeline.llm_with_backend.predictive_delegation for how it works. Pass one to LLMWithBackend(predictive_delegation=...), which puts it just in front of the frontend; it needs a text frontend and a request strategy whose delegate call needs no arguments, such as the default TranscriptBackendRequestStrategy.

It races every turn that ends with a new user message, spoken or typed, and so also a user-role message the app appends and runs the frontend on. The classifier is asked whether that message needs the backend, given the frontend’s system instruction (where the app says what the backend is for, and the pairing instruction when to delegate), the delegate tool’s description, and the latest messages of the conversation, the backend’s included. A turn the frontend sees as speculative (eager end of turn), or with incomplete-turn filtering on, is left to the frontend.

Example:

llm = LLMWithBackend(
    frontend=OpenAILLMService(...),
    backend=BackendLLMWorker(...),
    predictive_delegation=PredictiveDelegation(JevClassifier(api_key=...)),
)
__init__(classifier: BaseClassifier, **kwargs)[source]

Initialize predictive delegation.

Parameters:
  • classifier – The yes/no classifier to ask. It answers within a bound of its own; an answer that comes after the frontend’s first output is ignored.

  • **kwargs – Additional arguments passed to the parent class.

async setup(setup: FrameProcessorSetup)[source]

Set up the classifier, reporting its metrics in the pipeline.

async cleanup()[source]

Stop any classification in progress and clean up the classifier.

async process_frame(frame: Frame, direction: FrameDirection)[source]

Race each new user turn; pass every frame on.

Parameters:
  • frame – The frame to process.

  • direction – The direction of frame processing.

class pipecat.pipeline.llm_with_backend.TranscriptBackendRequestStrategy(*, frontend_instruction: str | None = 'A delegation hands over the whole conversation, not one item: the backend reads it and does everything in it that is its job, so delegate once per reply however many things the user asked for, and do not word the request.', tool_description: str = "The only way to start, change or stop the backend's work. Hands the conversation over to the backend, which reads it and does everything in it that needs a backend tool or careful reasoning, and stops or changes what the user has called off or changed. Call it as soon as any part of what the user says concerns that work, even while the backend is still working, and do the rest yourself. Not for a question about how the work is going, which you answer from the backend's notes. One handoff per reply, however many things the user asked for. It takes no arguments. Keep talking with the user while it works.", request_instruction: str = "Act on the user's most recent request in the conversation above. If it asks for something, tell the user the result as soon as you have it, before going on with other work, in a message that begins with >>. If it only stops or changes work already under way, tell them nothing: the assistant already has.")[source]

Bases: BackendRequestStrategy

Hands the backend the conversation and lets it work out the request.

The frontend model calls the tool with no arguments; the request is the transcript of what was said since the previous delegation (the whole conversation the first time), rendered by _render_transcript_request(). The backend’s own messages in the conversation are left out: the backend already has what it said. The default for a text frontend, whose context is current when the tool runs.

The cursor assumes the conversation accrues. A rewritten context or a failed delegation’s turns are not re-sent; the user’s next request carries what matters. A call that finds no new turn since the previous delegation sends nothing.

requires_context: bool = True

Whether the strategy reads the conversation from the frontend’s context, so it needs that context to be current when the delegate tool runs. A speech-to-speech frontend’s context lags the audio, so the connector refuses such a strategy there.

delegate_needs_arguments: bool = False

Whether a delegate call has to carry arguments for the request to be built. Predictive delegation makes its delegate call with none, so it works only with a strategy that sets this False.

__init__(*, frontend_instruction: str | None = 'A delegation hands over the whole conversation, not one item: the backend reads it and does everything in it that is its job, so delegate once per reply however many things the user asked for, and do not word the request.', tool_description: str = "The only way to start, change or stop the backend's work. Hands the conversation over to the backend, which reads it and does everything in it that needs a backend tool or careful reasoning, and stops or changes what the user has called off or changed. Call it as soon as any part of what the user says concerns that work, even while the backend is still working, and do the rest yourself. Not for a question about how the work is going, which you answer from the backend's notes. One handoff per reply, however many things the user asked for. It takes no arguments. Keep talking with the user while it works.", request_instruction: str = "Act on the user's most recent request in the conversation above. If it asks for something, tell the user the result as soon as you have it, before going on with other work, in a message that begins with >>. If it only stops or changes work already under way, tell them nothing: the assistant already has.")[source]

Initialize the strategy.

Parameters:
  • frontend_instruction – The strategy’s guidance to the frontend model, placed in the pairing instruction: that a delegation hands over the whole conversation, unworded.

  • tool_description – How the delegate tool is described.

  • request_instruction – What the backend should do with the transcript, placed after it.

async compose_request(params: FunctionCallParams) → str | None[source]

Render the turns since the previous delegation as the request.

Submodules