llm
Anthropic AI service integration for Pipecat.
This module provides LLM services and context management for Anthropic’s Claude models, including support for function calling, vision, and prompt caching features.
- class pipecat.services.anthropic.llm.AnthropicThinkingConfig(*, type: Literal['adaptive', 'enabled', 'disabled', 'between_tools'] | str, budget_tokens: int | None = None, display: Literal['summarized', 'omitted'] | str | None = None)[source]
Bases:
BaseModelConfiguration for thinking.
- Parameters:
type – Thinking mode. “adaptive” lets the model decide when and how deeply to think; prefer it. “enabled” is legacy manual thinking, sized by
budget_tokens: Claude 4.7 and later reject it, and Claude 4.5 and earlier accept only it. “disabled” turns thinking off on the models that allow it. “between_tools” is how Claude Sonnet 5.5, which rejects “disabled”, runs without up-front thinking; no other model accepts it.budget_tokens – Maximum number of tokens for thinking. With today’s models, the minimum is 1024. Required when type is “enabled”, not allowed otherwise.
display – How thinking text comes back: “summarized” for readable thinking, which is what
LLMThoughtTextFramecarries, or “omitted” for thinking blocks whose text is empty. Claude 4.7 and later default to “omitted”, so set “summarized” there to keep those frames carrying text. Not allowed when type is “disabled”.
- class pipecat.services.anthropic.llm.AnthropicLLMSettings(model: str | None | NotGiven = <factory>, extra: dict[str, Any]=<factory>, system_instruction: str | None | NotGiven = <factory>, temperature: float | None | NotGiven | NotGiven = <factory>, max_tokens: int | None | NotGiven = <factory>, top_p: float | None | NotGiven | NotGiven = <factory>, top_k: int | None | NotGiven | NotGiven = <factory>, frequency_penalty: float | None | NotGiven = <factory>, presence_penalty: float | None | NotGiven = <factory>, seed: int | None | NotGiven = <factory>, filter_incomplete_user_turns: bool | None | NotGiven = <factory>, user_turn_completion_config: UserTurnCompletionConfig | None | NotGiven = <factory>, use_provider_thinking_default: bool | NotGiven = <factory>, enable_prompt_caching: bool | NotGiven = <factory>, system_prompt_cache_ttl: Literal['5m', '1h'] | None | ~pipecat.utils.types.NotGiven=<factory>, thinking: AnthropicLLMService.ThinkingConfig | NotGiven | NotGiven = <factory>, effort: Literal['low', 'medium', 'high', 'xhigh', 'max'] | str | None | ~pipecat.utils.types.NotGiven=<factory>)[source]
Bases:
LLMSettingsSettings for AnthropicLLMService.
- Parameters:
enable_prompt_caching – Whether to enable prompt caching.
system_prompt_cache_ttl – Lifetime of the system prompt’s cache entry when prompt caching is enabled: “5m” or “1h”.
Noneuses Anthropic’s default of 5 minutes. “1h” keeps a system prompt shared by many conversations cached across gaps between them, at twice the base input price per cache write instead of 1.25 times. Anthropic caches nothing when the tools and system prompt together fall below the model’s minimum cacheable prompt length.thinking – Thinking configuration. If neither this nor
effortis set, Pipecat sends the lowest thinking setting Anthropic documents for the model, for the fastest responses, and logs what it sent. Models Pipecat doesn’t list, and any model whenuse_provider_thinking_defaultis set, run at Anthropic’s default.effort – How much the model spends on its response, thinking included: “low”, “medium”, “high”, “xhigh” or “max”. Sent as
output_config.effort.Noneuses the model’s default. Lower effort is Anthropic’s recommended way to reduce thinking, and the only one on models whose thinking can’t be turned off.
- classmethod from_mapping(settings)[source]
Convert a plain dict to settings, coercing thinking dicts.
For backward compatibility, a
thinkingvalue that is a plain dict is converted to aAnthropicLLMService.ThinkingConfig.
- class pipecat.services.anthropic.llm.AnthropicLLMService(*, api_key: str, model: str | None = None, params: InputParams | None = None, settings: AnthropicLLMSettings | None = None, client=None, retry_timeout_secs: float | None = 5.0, retry_on_timeout: bool | None = False, **kwargs)[source]
Bases:
LLMService[AnthropicLLMAdapter]LLM service for Anthropic’s Claude models.
Provides inference capabilities with Claude models including support for function calling, vision processing, streaming responses, and prompt caching. Can use custom clients like AsyncAnthropicBedrock and AsyncAnthropicVertex.
- Settings
alias of
AnthropicLLMSettings
- adapter_class
alias of
AnthropicLLMAdapter
- supports_response_schema: bool = True
Whether the provider can enforce a response schema in
run_inference().Services whose provider can set this to
True. When only some of its models can, they also overridemodel_supports_response_schema().
- ThinkingConfig
alias of
AnthropicThinkingConfig
- class InputParams(**kwargs)[source]
Bases:
BaseModelInput parameters for Anthropic model inference.
Deprecated since version 0.0.105: Use
AnthropicLLMService.Settingsinstead. Pass settings directly via thesettingsparameter ofAnthropicLLMService. Will be removed in 2.0.0.- Parameters:
enable_prompt_caching – Whether to enable the prompt caching feature.
max_tokens – Maximum tokens to generate. Must be at least 1.
temperature – Sampling temperature between 0.0 and 1.0.
top_k – Top-k sampling parameter.
top_p – Top-p sampling parameter between 0.0 and 1.0.
thinking – Extended thinking configuration. Enabling extended thinking causes the model to spend more time “thinking” before responding. It also causes this service to emit LLMThinking*Frames during response generation. If this is not provided, Pipecat sends the lowest thinking setting Anthropic documents for the model, for the fastest responses, and logs what it sent. Models Pipecat doesn’t list run at Anthropic’s default.
extra – Additional parameters to pass to the API.
- thinking: AnthropicThinkingConfig | None
- __init__(*, api_key: str, model: str | None = None, params: InputParams | None = None, settings: AnthropicLLMSettings | None = None, client=None, retry_timeout_secs: float | None = 5.0, retry_on_timeout: bool | None = False, **kwargs)[source]
Initialize the Anthropic LLM service.
- Parameters:
api_key – Anthropic API key for authentication.
model –
Model name to use.
Deprecated since version 0.0.105: Use
settings=AnthropicLLMService.Settings(model=...)instead. Will be removed in 2.0.0.params –
Optional model parameters for inference.
Deprecated since version 0.0.105: Use
settings=AnthropicLLMService.Settings(...)instead. Will be removed in 2.0.0.settings – Runtime-updatable settings for this service. When both deprecated parameters and settings are provided, settings values take precedence.
client – Optional custom Anthropic client instance.
retry_timeout_secs – Request timeout in seconds for retry logic.
retry_on_timeout – Whether to retry the request once if it times out.
**kwargs – Additional arguments passed to parent LLMService.
- can_generate_metrics() bool[source]
Check if this service can generate usage metrics.
- Returns:
True, as Anthropic provides detailed token usage metrics.
- static model_supports_response_schema(model: str) bool[source]
Whether a model can enforce a response schema.
Structured outputs arrived with the 4.5 models. A model id without a version, such as a preview, is assumed to support them.
- Parameters:
model – The model name.
- async run_inference(context: LLMContext, max_tokens: int | None = None, system_instruction: str | None = None, response_schema: dict[str, Any] | None = None) str | None[source]
Run a one-shot, out-of-band (i.e. out-of-pipeline) inference with the given LLM context.
- Parameters:
context – The LLM context containing conversation history.
max_tokens – Optional maximum number of tokens to generate. If provided, overrides the service’s default max_tokens setting.
system_instruction – Optional system instruction to use for this inference. If provided, overrides any system instruction in the context.
response_schema – Optional JSON schema the reply must follow. The service asks the provider to enforce it, so the reply is JSON text matching the schema.
- Returns:
The LLM’s response as a string, or None if no response is generated.
- async process_frame(frame: Frame, direction: FrameDirection)[source]
Process incoming frames and route them appropriately.
Handles various frame types including context frames, message frames, vision frames, and settings updates.
- Parameters:
frame – The frame to process.
direction – The direction of frame processing.