llm_with_backend
An LLM with a backend: a conversational frontend delegating the rest.
The frontend is any LLM service, text or speech-to-speech, and holds the
conversation. The backend is a BackendLLMWorker
running a heavier model with the tools, and does the work the frontend hands
off. LLMWithBackend wraps the frontend so the pair drops into a
pipeline where an LLM goes, and installs the tools that join them.
The two exchange messages. The frontend’s delegate tool puts a message
to the backend and returns at once; what the backend has to say comes back
as a stream of outputs, each appended to the frontend’s conversation as a
message marked Backend:, spoken or silent as the backend’s flag says. A
second request joins the first, and a correction changes or stops it. How a
request is worded is the BackendRequestStrategy’s business, and the
BackendConnector owns the tools, the session with the backend, and
how each output is rendered into the conversation.
The classes an app builds with import from this package. The rest, such as the prompt texts an app may start from to write its own, import from the module that defines them:
prompts: what the frontend’s model reads, from the pairing instruction to the marks on the backend’s messages.request_strategies: how adelegatecall becomes the backend’s request.connector:BackendConnector, which joins a frontend to its backend.predictive_delegation:PredictiveDelegation, an optional speed-up for a text frontend.llm_with_backend:LLMWithBackend, which puts the pair where an LLM goes.
- class pipecat.pipeline.llm_with_backend.BackendConnector(*, request_strategy: BackendRequestStrategy | None = None, frontend_pairing_instruction: str = 'THE BACKEND: You have a backend for anything that needs a backend tool or careful reasoning. It sees nothing of the conversation but what you delegate, and nothing you say reaches it: to start, change or stop its work, call the delegate tool, at once, in the same reply, and do the rest of the reply yourself (if any is needed) around it. That holds for every message that concerns its work, whether a new request, a yes to something that was offered, a correction or a "never mind", and even while earlier work is still running: a stop you only speak leaves the work running. Do not delegate what you can do yourself: small talk, a question that a result you already have answers, or a question about how the work is going, which you answer from the backend\'s latest notes or, with none yet, by saying there is no update yet. Never guess a result while waiting. {strategy}\n\nONE ASSISTANT: To the user there is one assistant, you: never mention the backend, delegation, a handoff or tools. Say what is being done as your own doing, such as "I\'m checking", never "the backend is checking", and never that you passed anything along.\n\nAFTER DELEGATING: Acknowledge in one short sentence of your own that fits the request, naming the task briefly if that reads naturally. For example: "Let me look into that", "Sure, I\'ll get on that now" or "Okay, pulling that up now". If you already acknowledged alongside the delegate call, even in part, the user has heard it, and this reply should *not* acknowledge again (though it can "finish the thought" of the previous acknowledgement, if appropriate). Saying it in other words is repeating yourself, and so is adding that you are on it now, that it will take a moment, or that you will report back when it is done; the initial acknowledgement already said all of that. After "Let me look that up for you!", e.g., there is nothing left to say, so say nothing. Finish the thought only when the user asked for something else that you have not answered yet. An acknowledgement is never a substitute for the delegate call. In the acknowledgement, avoid restating the request in full, or describing in detail what you are going to do, or following up on your own with promises, offers or questions: you\'ve fired off the delegate call, so let it be. Then answer whatever else the user asked that you can. Never say work is done, or give a result, that the backend has not given you; if the user asks whether work is under way and you never delegated it, say so and delegate it then.\n\nBACKEND MESSAGES: The backend works on its own after you delegate and may be doing several things at once. What it has to say arrives as messages in the conversation marked "Backend:": a result, a question for the user, or news worth an update, on whatever turn happens to be in progress by then. Such a message is owed to the user, whatever the conversation has moved on to: small talk, other questions or a long wait do not make it unwanted. If the user has said something you have not answered yet, answer that first, then relay the message at the end of that same reply, in your own words. If you have already answered everything the user said, relay just the message; do not repeat or rephrase your earlier reply. Relay each message once. Only when the user has cancelled or changed what they asked for does a message go unsaid, and then say only what still helps. Messages marked "Backend (working):", "Backend (note):" and "Backend (thinking):" are what the backend is doing and thinking. They carry no results: what a call found reaches you only in a message marked "Backend:", so never describe or guess what a call found. Never relay them on their own, but when the user asks how the work is going, answer from the latest of them, concretely: name the step, such as which file it is reading or that it is running the tests, rather than saying only that it is still working. Never state a result you have not received from the backend, and never say work is done that the backend has not said is done.\n\nWhatever else a reply does, a user message about work you delegated, whether to add to it, change it or stop it, a never mind included, gets a delegate call in that reply: nothing else stops or changes that work.', respond_on_delegate: bool = True, timeout_secs: float | None = 30, client_trace: bool = False)[source]
Bases:
objectJoins the two layers: the tools the frontend calls, the session with the backend, and what each output becomes.
A request strategy defines the
delegatetool and composes the request. It may be given, or left to the connector to pick by frontend kind once bound:TranscriptBackendRequestStrategyfor a text frontend,ExplicitBackendRequestStrategyfor a speech-to-speech one, whose context lags the audio. A given strategy that reads the frontend’s context (BackendRequestStrategy.requires_context) is refused for a speech-to-speech frontend.The connector holds the session with the backend for the frontend’s life, and turns each output the backend sends into a message appended to the frontend’s conversation with
render_output(), run or not as the output’sprefers_spokenflag says. The function calls the backend makes on the way are reported in the frontend’s pipeline for clients to show; nothing else in the pipeline sees them.Example:
connector = BackendConnector(respond_on_delegate=False)
- __init__(*, request_strategy: BackendRequestStrategy | None = None, frontend_pairing_instruction: str = 'THE BACKEND: You have a backend for anything that needs a backend tool or careful reasoning. It sees nothing of the conversation but what you delegate, and nothing you say reaches it: to start, change or stop its work, call the delegate tool, at once, in the same reply, and do the rest of the reply yourself (if any is needed) around it. That holds for every message that concerns its work, whether a new request, a yes to something that was offered, a correction or a "never mind", and even while earlier work is still running: a stop you only speak leaves the work running. Do not delegate what you can do yourself: small talk, a question that a result you already have answers, or a question about how the work is going, which you answer from the backend\'s latest notes or, with none yet, by saying there is no update yet. Never guess a result while waiting. {strategy}\n\nONE ASSISTANT: To the user there is one assistant, you: never mention the backend, delegation, a handoff or tools. Say what is being done as your own doing, such as "I\'m checking", never "the backend is checking", and never that you passed anything along.\n\nAFTER DELEGATING: Acknowledge in one short sentence of your own that fits the request, naming the task briefly if that reads naturally. For example: "Let me look into that", "Sure, I\'ll get on that now" or "Okay, pulling that up now". If you already acknowledged alongside the delegate call, even in part, the user has heard it, and this reply should *not* acknowledge again (though it can "finish the thought" of the previous acknowledgement, if appropriate). Saying it in other words is repeating yourself, and so is adding that you are on it now, that it will take a moment, or that you will report back when it is done; the initial acknowledgement already said all of that. After "Let me look that up for you!", e.g., there is nothing left to say, so say nothing. Finish the thought only when the user asked for something else that you have not answered yet. An acknowledgement is never a substitute for the delegate call. In the acknowledgement, avoid restating the request in full, or describing in detail what you are going to do, or following up on your own with promises, offers or questions: you\'ve fired off the delegate call, so let it be. Then answer whatever else the user asked that you can. Never say work is done, or give a result, that the backend has not given you; if the user asks whether work is under way and you never delegated it, say so and delegate it then.\n\nBACKEND MESSAGES: The backend works on its own after you delegate and may be doing several things at once. What it has to say arrives as messages in the conversation marked "Backend:": a result, a question for the user, or news worth an update, on whatever turn happens to be in progress by then. Such a message is owed to the user, whatever the conversation has moved on to: small talk, other questions or a long wait do not make it unwanted. If the user has said something you have not answered yet, answer that first, then relay the message at the end of that same reply, in your own words. If you have already answered everything the user said, relay just the message; do not repeat or rephrase your earlier reply. Relay each message once. Only when the user has cancelled or changed what they asked for does a message go unsaid, and then say only what still helps. Messages marked "Backend (working):", "Backend (note):" and "Backend (thinking):" are what the backend is doing and thinking. They carry no results: what a call found reaches you only in a message marked "Backend:", so never describe or guess what a call found. Never relay them on their own, but when the user asks how the work is going, answer from the latest of them, concretely: name the step, such as which file it is reading or that it is running the tests, rather than saying only that it is still working. Never state a result you have not received from the backend, and never say work is done that the backend has not said is done.\n\nWhatever else a reply does, a user message about work you delegated, whether to add to it, change it or stop it, a never mind included, gets a delegate call in that reply: nothing else stops or changes that work.', respond_on_delegate: bool = True, timeout_secs: float | None = 30, client_trace: bool = False)[source]
Initialize the connector.
- Parameters:
request_strategy – How a
delegatecall becomes the backend’s request. Picked by frontend kind when omitted.frontend_pairing_instruction – How the model works with a backend, appended to its system instruction:
FRONTEND_PAIRING_INSTRUCTIONunless given (both constants named here are inprompts). One of its own has to say what that one says: when and how to delegate, that to the user there is one assistant, what to do after delegating, and what to do with the backend’s messages. The request strategy’s guidance goes wherePAIRING_INSTRUCTION_STRATEGY_PLACEHOLDERstands, or after the text when it has none.respond_on_delegate – Whether the frontend runs again as soon as a delegation is sent, which is where its acknowledgement comes from: a text model’s tool-call turn carries no prose. Off for a frontend that announces a handoff itself.
timeout_secs – How long the backend may take to acknowledge a message, including the wait for it to become ready.
client_trace – Whether to send every exchange with the backend to the client as an RTVI server message (
type"llm-with-backend"): each request, output, tool-call phase, error and idle. For debugging; a client’s event log shows them.
- property request_strategy: BackendRequestStrategy
The request strategy in use. Available once bound.
- property tools: list[FunctionSchema]
The tools to install on the frontend. Available once bound.
- property tool: FunctionSchema
The
delegatetool. Available once bound.
- property frontend_instruction: str | None
the pairing instruction with the strategy’s guidance in place. Available once bound.
- Type:
What is appended to the frontend’s system instruction
- bind(context: ConnectorContext) None[source]
Settle the strategy for the layers being joined and build the tools.
- Parameters:
context – The layers.
- Raises:
ValueError – If the frontend is speech-to-speech and the strategy reads the frontend’s context, which lags the audio there.
- build_tools() list[FunctionSchema][source]
Build the tools to install on the frontend:
delegate.Override to install tools of another shape entirely; the service reads nothing else from the connector but
frontend_instruction.- Returns:
The tools, each carrying its handler.
- async run_session(worker: BaseWorker, frontend: LLMService[Any]) None[source]
Hold the session with the backend, delivering what it produces, until cancelled.
- Parameters:
worker – The pipeline worker the frontend runs in.
frontend – The frontend service, which takes the deliveries.
- async delegate(params: FunctionCallParams) None[source]
Put the request a
delegatecall carries to the backend, and settle the call at once.- Parameters:
params – The
delegatecall.
- async deliver(frontend: LLMService[Any], event: BackendOutput | BackendToolCall | BackendIdle | BackendError) None[source]
Deliver one thing the backend produced to the frontend.
An output becomes a message appended to the frontend’s conversation, which runs the frontend when the output asks to be spoken. A function call phase is reported as the
ExternalFunctionCall*Framefor it, which the RTVI observer turns into function-call events. An error becomes a spoken message, so the user hears the work stopped. The appended messages are uninterruptible: one queued while the bot speaks waits behind that speech, and an interruption clears the speech but must leave the message to be recorded.- Parameters:
frontend – The frontend service.
event – What the backend produced.
- render_output(output: BackendOutput) Any | LLMSpecificMessage | None[source]
Render a backend output as the message the frontend’s conversation takes in.
Override for another wording, role, or to leave some outputs out by returning
None. The default marks the message so the frontend can tell it from the user’s and its own, and so the transcript request strategy can leave it out of what it sends the backend: a spoken output asBackend:withBACKEND_MESSAGE_RELAY_INSTRUCTIONafter it, a silent one asBackend (note):, a reasoning summary asBackend (thinking):.- Parameters:
output – The output.
- Returns:
The message, or
Noneto deliver nothing for this output.
- render_tool_call(call: BackendToolCall) Any | LLMSpecificMessage | None[source]
Render a phase of a backend function call as a message the frontend’s conversation takes in, silently.
The default records each call as it starts, with its arguments, so the frontend can say what the backend is doing if asked; the other phases are not recorded. Override to record more, less, or nothing.
- Parameters:
call – The phase of the call.
- Returns:
The message, or
Noneto record nothing for this phase.
- class pipecat.pipeline.llm_with_backend.BackendRequestStrategy[source]
Bases:
objectDefines the
delegatetool the frontend calls, and turns a call into the request the backend receives.The tool’s parameters are this strategy’s to declare, so the connector’s handler has one signature and reads whatever was declared back from
params.argumentsincompose_request().- tool_parameters: dict[str, Any] = {}
JSON-schema properties of the
delegatetool; empty when it takes none.
- frontend_instruction: str | None = None
how the request is worded. It goes where the pairing instruction’s
PAIRING_INSTRUCTION_STRATEGY_PLACEHOLDERstands.- Type:
The strategy’s guidance to the frontend model, if any
- requires_context: bool = False
Whether the strategy reads the conversation from the frontend’s context, so it needs that context to be current when the
delegatetool runs. A speech-to-speech frontend’s context lags the audio, so the connector refuses such a strategy there.
- delegate_needs_arguments: bool = True
Whether a
delegatecall has to carry arguments for the request to be built. Predictive delegation makes itsdelegatecall with none, so it works only with a strategy that sets this False.
- async compose_request(params: FunctionCallParams) str | None[source]
Turn a
delegatecall into the text put to the backend.- Parameters:
params – The call, carrying the arguments the tool declared and the frontend’s context.
- Returns:
The request text, or
Nonewhen the call carries nothing new for the backend, in which case nothing is sent.
- class pipecat.pipeline.llm_with_backend.ExplicitBackendRequestStrategy(*, frontend_instruction: str | None = "Word the request so it stands on its own: the user's goal, the exact details they gave and their latest correction, with everything new they asked for in the one request, so you delegate once per reply however many things that is. Work already handed over stays with the backend: a new request joins it, so do not restate earlier requests; a correction names what changes, and a stop names what to stop.", tool_description: str = "The only way to start, change or stop the backend's work. Hands a request to the backend, which does everything in it that needs a backend tool or careful reasoning, and stops or changes what the user has called off or changed. Call it as soon as any part of what the user says concerns that work, even while the backend is still working, with the request worded to stand on its own, and do the rest yourself. Not for a question about how the work is going, which you answer from the backend's notes. One call per reply, however many things the user asked for. Work already handed over stays with the backend, so send only what is new, including anything the user says about that work, such as to stop or change it. Keep talking with the user while it works.", request_instruction: str = 'If this request asks for something, tell the user the result as soon as you have it, before going on with other work, in a message that begins with >>. If it only stops or changes work already under way, tell them nothing: the assistant already has.')[source]
Bases:
BackendRequestStrategyHas the frontend model word the request itself.
The tool takes a
requestargument, which is sent to the backend as it stands. The default for a speech-to-speech frontend, which responds to audio before the transcript of that audio reaches its context, so the turns that prompted a handoff may not be recorded when the tool runs.- tool_parameters: dict[str, Any] = {'request': {'description': 'What the user is asking for now, self-contained: their goal, the details they gave (places, dates, names) and their latest correction, or what they said about work already under way, such as to stop it or change it. Work already handed over stays with the backend; a new request joins it, so do not restate earlier requests.', 'type': 'string'}}
JSON-schema properties of the
delegatetool; empty when it takes none.
- __init__(*, frontend_instruction: str | None = "Word the request so it stands on its own: the user's goal, the exact details they gave and their latest correction, with everything new they asked for in the one request, so you delegate once per reply however many things that is. Work already handed over stays with the backend: a new request joins it, so do not restate earlier requests; a correction names what changes, and a stop names what to stop.", tool_description: str = "The only way to start, change or stop the backend's work. Hands a request to the backend, which does everything in it that needs a backend tool or careful reasoning, and stops or changes what the user has called off or changed. Call it as soon as any part of what the user says concerns that work, even while the backend is still working, with the request worded to stand on its own, and do the rest yourself. Not for a question about how the work is going, which you answer from the backend's notes. One call per reply, however many things the user asked for. Work already handed over stays with the backend, so send only what is new, including anything the user says about that work, such as to stop or change it. Keep talking with the user while it works.", request_instruction: str = 'If this request asks for something, tell the user the result as soon as you have it, before going on with other work, in a message that begins with >>. If it only stops or changes work already under way, tell them nothing: the assistant already has.')[source]
Initialize the strategy.
- Parameters:
frontend_instruction – The strategy’s guidance to the frontend model, placed in the pairing instruction: how to word the request.
tool_description – How the
delegatetool is described.request_instruction – What the backend should do with the request, placed after it.
- async compose_request(params: FunctionCallParams) str | None[source]
Send the model’s request as it stands, with the reporting expectation after it.
- class pipecat.pipeline.llm_with_backend.LLMWithBackend(*, frontend: LLMService[Any], backend: BackendLLMWorker | str, connector: BackendConnector | None = None, predictive_delegation: PredictiveDelegation | None = None)[source]
Bases:
PipelineA conversational LLM, kept fast and light, with a backend that does the work it hands off.
Alpha. The API is unstable: names, arguments, the strategies and the job contract are likely to change between releases.
Put it where the LLM goes in a pipeline. It is a
Pipeline, not anLLMService: functions, settings updates, event handlers and aFlowManagergo onfrontend. It wraps the frontend service, installs the connector’s tools on it as built-in tools, appends the connector’s guidance to the frontend’s system instruction, adds a local backend worker to the pipeline worker so the app never wires it up, and holds the session with the backend for the pipeline’s life. A built-in tool is sent on every inference beside whatever tools the frontend has, and never enters the context’s tool set, so a tool change announced to the model never mentions it. The tools are the backend’s; the frontend hasdelegateand no more, though a tool it must keep, in its context or configured on the service, stays.The guidance is the connector’s pairing instruction (how to work with a backend: when and how to delegate, and what to do with the backend’s messages) with the strategy’s (how the request is worded) in place. The frontend’s own system instruction is the place to say what the backend is for, in plain words, as the backend’s own instruction does: no tool names, nothing to update when a tool changes.
Example:
llm = LLMWithBackend( frontend=OpenAILLMService(...), backend=BackendLLMWorker( llm=AnthropicLLMService(...), context=LLMContext(tools=[get_weather]), ), ) pipeline = Pipeline([transport.input(), stt, user_agg, llm, tts, transport.output(), assistant_agg])
- __init__(*, frontend: LLMService[Any], backend: BackendLLMWorker | str, connector: BackendConnector | None = None, predictive_delegation: PredictiveDelegation | None = None)[source]
Initialize the service.
- Parameters:
frontend – The conversational LLM service, text or speech-to-speech.
backend – The backend worker, or the name of one registered elsewhere (in the app, or in another process on a shared bus). A worker given here is added to the pipeline worker at setup.
connector – How delegation works. A default
BackendConnectorwhen omitted.predictive_delegation – A
PredictiveDelegation, to speed up responses that need backend involvement: its classifier determines whether to wait for the frontend’s inference or go ahead and trigger an early backend delegation. Only for a text frontend.
- Raises:
ValueError – If the frontend service declines the role; if the connector’s request strategy reads the frontend’s context and the frontend is speech-to-speech; or if predictive delegation is given with a speech-to-speech frontend or with a request strategy whose
delegatecall needs arguments.
- property frontend: LLMService[Any]
The frontend LLM service.
- property backend: BackendLLMWorker | str
The backend worker, or its name.
- property connector: BackendConnector
The connector joining the two layers.
- property predictive_delegation: PredictiveDelegation | None
The PredictiveDelegation in front of the frontend, if there is one.
- property tool: FunctionSchema
The
delegatetool installed on the frontend.
- async setup(setup: FrameProcessorSetup)[source]
Register a local backend worker as a child of the pipeline worker, set up the frontend, and open the session.
- class pipecat.pipeline.llm_with_backend.PredictiveDelegation(classifier: BaseClassifier, **kwargs)[source]
Bases:
FrameProcessorAn optimization that can speed up responses that need backend involvement.
Alpha. It uses a fast yes/no classifier to determine, after each user request, whether it needs to wait for the frontend’s inference or can go ahead and trigger an early backend delegation. See
pipecat.pipeline.llm_with_backend.predictive_delegationfor how it works. Pass one toLLMWithBackend(predictive_delegation=...), which puts it just in front of the frontend; it needs a text frontend and a request strategy whosedelegatecall needs no arguments, such as the defaultTranscriptBackendRequestStrategy.It races every turn that ends with a new user message, spoken or typed, and so also a user-role message the app appends and runs the frontend on. The classifier is asked whether that message needs the backend, given the frontend’s system instruction (where the app says what the backend is for, and the pairing instruction when to delegate), the
delegatetool’s description, and the latest messages of the conversation, the backend’s included. A turn the frontend sees as speculative (eager end of turn), or with incomplete-turn filtering on, is left to the frontend.Example:
llm = LLMWithBackend( frontend=OpenAILLMService(...), backend=BackendLLMWorker(...), predictive_delegation=PredictiveDelegation(JevClassifier(api_key=...)), )
- __init__(classifier: BaseClassifier, **kwargs)[source]
Initialize predictive delegation.
- Parameters:
classifier – The yes/no classifier to ask. It answers within a bound of its own; an answer that comes after the frontend’s first output is ignored.
**kwargs – Additional arguments passed to the parent class.
- async setup(setup: FrameProcessorSetup)[source]
Set up the classifier, reporting its metrics in the pipeline.
- async process_frame(frame: Frame, direction: FrameDirection)[source]
Race each new user turn; pass every frame on.
- Parameters:
frame – The frame to process.
direction – The direction of frame processing.
- class pipecat.pipeline.llm_with_backend.TranscriptBackendRequestStrategy(*, frontend_instruction: str | None = 'A delegation hands over the whole conversation, not one item: the backend reads it and does everything in it that is its job, so delegate once per reply however many things the user asked for, and do not word the request.', tool_description: str = "The only way to start, change or stop the backend's work. Hands the conversation over to the backend, which reads it and does everything in it that needs a backend tool or careful reasoning, and stops or changes what the user has called off or changed. Call it as soon as any part of what the user says concerns that work, even while the backend is still working, and do the rest yourself. Not for a question about how the work is going, which you answer from the backend's notes. One handoff per reply, however many things the user asked for. It takes no arguments. Keep talking with the user while it works.", request_instruction: str = "Act on the user's most recent request in the conversation above. If it asks for something, tell the user the result as soon as you have it, before going on with other work, in a message that begins with >>. If it only stops or changes work already under way, tell them nothing: the assistant already has.")[source]
Bases:
BackendRequestStrategyHands the backend the conversation and lets it work out the request.
The frontend model calls the tool with no arguments; the request is the transcript of what was said since the previous delegation (the whole conversation the first time), rendered by
_render_transcript_request(). The backend’s own messages in the conversation are left out: the backend already has what it said. The default for a text frontend, whose context is current when the tool runs.The cursor assumes the conversation accrues. A rewritten context or a failed delegation’s turns are not re-sent; the user’s next request carries what matters. A call that finds no new turn since the previous delegation sends nothing.
- requires_context: bool = True
Whether the strategy reads the conversation from the frontend’s context, so it needs that context to be current when the
delegatetool runs. A speech-to-speech frontend’s context lags the audio, so the connector refuses such a strategy there.
- delegate_needs_arguments: bool = False
Whether a
delegatecall has to carry arguments for the request to be built. Predictive delegation makes itsdelegatecall with none, so it works only with a strategy that sets this False.
- __init__(*, frontend_instruction: str | None = 'A delegation hands over the whole conversation, not one item: the backend reads it and does everything in it that is its job, so delegate once per reply however many things the user asked for, and do not word the request.', tool_description: str = "The only way to start, change or stop the backend's work. Hands the conversation over to the backend, which reads it and does everything in it that needs a backend tool or careful reasoning, and stops or changes what the user has called off or changed. Call it as soon as any part of what the user says concerns that work, even while the backend is still working, and do the rest yourself. Not for a question about how the work is going, which you answer from the backend's notes. One handoff per reply, however many things the user asked for. It takes no arguments. Keep talking with the user while it works.", request_instruction: str = "Act on the user's most recent request in the conversation above. If it asks for something, tell the user the result as soon as you have it, before going on with other work, in a message that begins with >>. If it only stops or changes work already under way, tell them nothing: the assistant already has.")[source]
Initialize the strategy.
- Parameters:
frontend_instruction – The strategy’s guidance to the frontend model, placed in the pairing instruction: that a delegation hands over the whole conversation, unworded.
tool_description – How the
delegatetool is described.request_instruction – What the backend should do with the transcript, placed after it.
- async compose_request(params: FunctionCallParams) str | None[source]
Render the turns since the previous delegation as the request.
Submodules
- connector
DELEGATE_TOOL_NAMEConnectorContextBackendConnectorBackendConnector.__init__()BackendConnector.request_strategyBackendConnector.toolsBackendConnector.toolBackendConnector.frontend_instructionBackendConnector.sessionBackendConnector.bind()BackendConnector.build_tools()BackendConnector.run_session()BackendConnector.delegate()BackendConnector.deliver()BackendConnector.render_output()BackendConnector.render_tool_call()
- llm_with_backend
- predictive_delegation
- prompts
BACKEND_MESSAGE_PREFIXBACKEND_THOUGHT_PREFIXBACKEND_NOTE_PREFIXBACKEND_WORKING_PREFIXBACKEND_PREFIXESBACKEND_MESSAGE_RELAY_INSTRUCTIONPAIRING_INSTRUCTION_STRATEGY_PLACEHOLDERFRONTEND_PAIRING_INSTRUCTIONTRANSCRIPT_FRONTEND_INSTRUCTIONTRANSCRIPT_TOOL_DESCRIPTIONEXPLICIT_FRONTEND_INSTRUCTIONEXPLICIT_TOOL_DESCRIPTION
- request_strategies