llm

Anthropic AI service integration for Pipecat.

This module provides LLM services and context management for Anthropic’s Claude models, including support for function calling, vision, and prompt caching features.

class pipecat.services.anthropic.llm.AnthropicThinkingConfig(*, type: Literal['adaptive', 'enabled', 'disabled', 'between_tools'] | str, budget_tokens: int | None = None, display: Literal['summarized', 'omitted'] | str | None = None)[source]

Bases: BaseModel

Configuration for thinking.

Parameters:
  • type – Thinking mode. “adaptive” lets the model decide when and how deeply to think; prefer it. “enabled” is legacy manual thinking, sized by budget_tokens: Claude 4.7 and later reject it, and Claude 4.5 and earlier accept only it. “disabled” turns thinking off on the models that allow it. “between_tools” is how Claude Sonnet 5.5, which rejects “disabled”, runs without up-front thinking; no other model accepts it.

  • budget_tokens – Maximum number of tokens for thinking. With today’s models, the minimum is 1024. Required when type is “enabled”, not allowed otherwise.

  • display – How thinking text comes back: “summarized” for readable thinking, which is what LLMThoughtTextFrame carries, or “omitted” for thinking blocks whose text is empty. Claude 4.7 and later default to “omitted”, so set “summarized” there to keep those frames carrying text. Not allowed when type is “disabled”.

class pipecat.services.anthropic.llm.AnthropicLLMSettings(model: str | None | NotGiven = <factory>, extra: dict[str, Any]=<factory>, system_instruction: str | None | NotGiven = <factory>, temperature: float | None | NotGiven | NotGiven = <factory>, max_tokens: int | None | NotGiven = <factory>, top_p: float | None | NotGiven | NotGiven = <factory>, top_k: int | None | NotGiven | NotGiven = <factory>, frequency_penalty: float | None | NotGiven = <factory>, presence_penalty: float | None | NotGiven = <factory>, seed: int | None | NotGiven = <factory>, filter_incomplete_user_turns: bool | None | NotGiven = <factory>, user_turn_completion_config: UserTurnCompletionConfig | None | NotGiven = <factory>, use_provider_thinking_default: bool | NotGiven = <factory>, enable_prompt_caching: bool | NotGiven = <factory>, system_prompt_cache_ttl: Literal['5m', '1h'] | None | ~pipecat.utils.types.NotGiven=<factory>, thinking: AnthropicLLMService.ThinkingConfig | NotGiven | NotGiven = <factory>, effort: Literal['low', 'medium', 'high', 'xhigh', 'max'] | str | None | ~pipecat.utils.types.NotGiven=<factory>)[source]

Bases: LLMSettings

Settings for AnthropicLLMService.

Parameters:
  • enable_prompt_caching – Whether to enable prompt caching.

  • system_prompt_cache_ttl – Lifetime of the system prompt’s cache entry when prompt caching is enabled: “5m” or “1h”. None uses Anthropic’s default of 5 minutes. “1h” keeps a system prompt shared by many conversations cached across gaps between them, at twice the base input price per cache write instead of 1.25 times. Anthropic caches nothing when the tools and system prompt together fall below the model’s minimum cacheable prompt length.

  • thinking – Thinking configuration. If neither this nor effort is set, Pipecat sends the lowest thinking setting Anthropic documents for the model, for the fastest responses, and logs what it sent. Models Pipecat doesn’t list, and any model when use_provider_thinking_default is set, run at Anthropic’s default.

  • effort – How much the model spends on its response, thinking included: “low”, “medium”, “high”, “xhigh” or “max”. Sent as output_config.effort. None uses the model’s default. Lower effort is Anthropic’s recommended way to reduce thinking, and the only one on models whose thinking can’t be turned off.

classmethod from_mapping(settings)[source]

Convert a plain dict to settings, coercing thinking dicts.

For backward compatibility, a thinking value that is a plain dict is converted to a AnthropicLLMService.ThinkingConfig.

class pipecat.services.anthropic.llm.AnthropicLLMService(*, api_key: str, model: str | None = None, params: InputParams | None = None, settings: AnthropicLLMSettings | None = None, client=None, retry_timeout_secs: float | None = 5.0, retry_on_timeout: bool | None = False, **kwargs)[source]

Bases: LLMService[AnthropicLLMAdapter]

LLM service for Anthropic’s Claude models.

Provides inference capabilities with Claude models including support for function calling, vision processing, streaming responses, and prompt caching. Can use custom clients like AsyncAnthropicBedrock and AsyncAnthropicVertex.

Settings

alias of AnthropicLLMSettings

adapter_class

alias of AnthropicLLMAdapter

supports_response_schema: bool = True

Whether the provider can enforce a response schema in run_inference().

Services whose provider can set this to True. When only some of its models can, they also override model_supports_response_schema().

ThinkingConfig

alias of AnthropicThinkingConfig

class InputParams(**kwargs)[source]

Bases: BaseModel

Input parameters for Anthropic model inference.

Deprecated since version 0.0.105: Use AnthropicLLMService.Settings instead. Pass settings directly via the settings parameter of AnthropicLLMService. Will be removed in 2.0.0.

Parameters:
  • enable_prompt_caching – Whether to enable the prompt caching feature.

  • max_tokens – Maximum tokens to generate. Must be at least 1.

  • temperature – Sampling temperature between 0.0 and 1.0.

  • top_k – Top-k sampling parameter.

  • top_p – Top-p sampling parameter between 0.0 and 1.0.

  • thinking – Extended thinking configuration. Enabling extended thinking causes the model to spend more time “thinking” before responding. It also causes this service to emit LLMThinking*Frames during response generation. If this is not provided, Pipecat sends the lowest thinking setting Anthropic documents for the model, for the fastest responses, and logs what it sent. Models Pipecat doesn’t list run at Anthropic’s default.

  • extra – Additional parameters to pass to the API.

enable_prompt_caching: bool | None
max_tokens: int | None
temperature: float | None
top_k: int | None
top_p: float | None
thinking: AnthropicThinkingConfig | None
extra: dict[str, Any] | None
__init__(*, api_key: str, model: str | None = None, params: InputParams | None = None, settings: AnthropicLLMSettings | None = None, client=None, retry_timeout_secs: float | None = 5.0, retry_on_timeout: bool | None = False, **kwargs)[source]

Initialize the Anthropic LLM service.

Parameters:
  • api_key – Anthropic API key for authentication.

  • model –

    Model name to use.

    Deprecated since version 0.0.105: Use settings=AnthropicLLMService.Settings(model=...) instead. Will be removed in 2.0.0.

  • params –

    Optional model parameters for inference.

    Deprecated since version 0.0.105: Use settings=AnthropicLLMService.Settings(...) instead. Will be removed in 2.0.0.

  • settings – Runtime-updatable settings for this service. When both deprecated parameters and settings are provided, settings values take precedence.

  • client – Optional custom Anthropic client instance.

  • retry_timeout_secs – Request timeout in seconds for retry logic.

  • retry_on_timeout – Whether to retry the request once if it times out.

  • **kwargs – Additional arguments passed to parent LLMService.

can_generate_metrics() → bool[source]

Check if this service can generate usage metrics.

Returns:

True, as Anthropic provides detailed token usage metrics.

static model_supports_response_schema(model: str) → bool[source]

Whether a model can enforce a response schema.

Structured outputs arrived with the 4.5 models. A model id without a version, such as a preview, is assumed to support them.

Parameters:

model – The model name.

async run_inference(context: LLMContext, max_tokens: int | None = None, system_instruction: str | None = None, response_schema: dict[str, Any] | None = None) → str | None[source]

Run a one-shot, out-of-band (i.e. out-of-pipeline) inference with the given LLM context.

Parameters:
  • context – The LLM context containing conversation history.

  • max_tokens – Optional maximum number of tokens to generate. If provided, overrides the service’s default max_tokens setting.

  • system_instruction – Optional system instruction to use for this inference. If provided, overrides any system instruction in the context.

  • response_schema – Optional JSON schema the reply must follow. The service asks the provider to enforce it, so the reply is JSON text matching the schema.

Returns:

The LLM’s response as a string, or None if no response is generated.

async process_frame(frame: Frame, direction: FrameDirection)[source]

Process incoming frames and route them appropriately.

Handles various frame types including context frames, message frames, vision frames, and settings updates.

Parameters:
  • frame – The frame to process.

  • direction – The direction of frame processing.