string

Text processing utilities for sentence boundary detection and tag parsing.

This module provides utilities for natural language text processing including sentence boundary detection, email and number pattern handling, and XML-style tag parsing for structured text content.

Sentence boundaries use sentencex’s embedded language rules without external data.

pipecat.utils.string.resolve_sentence_tokenizer_language(language: str | None) → str[source]

Normalize a language code for sentencex’s language fallback map.

Parameters:

language – Language code, optionally with a region. None, empty, and automatic language selections use English.

Returns:

A lowercase base language code. Sentencex resolves unsupported codes.

pipecat.utils.string.replace_match(text: str, match: Match, old: str, new: str) → str[source]

Replace occurrences of a substring within a matched section of text.

Parameters:
  • text – The input text in which replacements will be made.

  • match – A regex match object representing the section of text to modify.

  • old – The substring to be replaced.

  • new – The substring to replace old with.

Returns:

The modified text with the specified replacements made within the matched section.

pipecat.utils.string.match_endofsentence(text: str, *, language: str | None = None) → int[source]

Find the position of the end of a sentence in the provided text.

This function uses sentencex’s sentence tokenizer to detect sentence boundaries in the input text, combined with punctuation verification to ensure that single tokens without proper sentence endings aren’t considered complete sentences.

Parameters:
  • text – The input text in which to find the end of the sentence.

  • language – Language code. Defaults to English.

Returns:

The position of the end of the sentence if found, otherwise 0.

pipecat.utils.string.parse_start_end_tags(text: str, tags: Sequence[tuple[str, str]], current_tag: tuple[str, str] | None, current_tag_index: int) → tuple[tuple[str, str] | None, int][source]

Parse text to identify start and end tag pairs.

Scans the text from current_tag_index: inside a tag, for its end tag; outside one, for the earliest start tag of any pair, and then for that pair’s end tag after it. A pair whose start and end tags are the same, such as ("^^", "^^"), is found the same way.

Parameters:
  • text – The text to be parsed.

  • tags – List of tuples containing start and end tags.

  • current_tag – The currently active tags, if any.

  • current_tag_index – Where in the text to resume scanning.

Returns:

just past the open tag’s start, or past the last tag that closed.

Return type:

The tag the text ends inside, or None, and where to resume scanning

pipecat.utils.string.longest_trailing_partial_match(text: str, candidates: Sequence[str]) → int[source]

Find the length of the longest suffix of text that is a proper prefix of a candidate.

Used to detect a delimiter (e.g., an XML-style start tag) that has been split across two text chunks: the trailing partial delimiter can be held back from output until the next chunk completes it, rather than being flushed as plain text and losing the delimiter.

Parameters:
  • text – The text to check for a trailing partial match.

  • candidates – Full strings to match a proper prefix of against the end of text.

Returns:

The length of the longest matching suffix, or 0 if no candidate has a proper prefix matching the end of text.

class pipecat.utils.string.TextPartForConcatenation(text: str, includes_inter_part_spaces: bool)[source]

Bases: object

Class representing a part of text for concatenation with concatenate_aggregated_text.

Parameters:
  • text – The text content.

  • includes_inter_part_spaces – Whether any necessary inter-frame (leading/trailing) spaces are already included in the text.

pipecat.utils.string.concatenate_aggregated_text(text_parts: list[TextPartForConcatenation]) → str[source]

Concatenate a list of text parts into a single string.

This function joins the provided list of text parts into a single string, taking into account whether or not the parts already contain spacing.

This function is useful for aggregating text segments received from LLMs or transcription services.

Parameters:

text_parts – A list of text parts to concatenate.

Returns:

A single concatenated string.

pipecat.utils.string.compile_phrase_pattern(phrase: str) → Pattern[source]

Compile a case-insensitive pattern that finds a spoken phrase in text.

The words of the phrase may be separated by any amount of whitespace (including none). The match must start and end on a word boundary, except next to characters of scripts written without spaces (Chinese, Japanese, Thai), where speech-to-text output has no boundaries between words.

Parameters:

phrase – The phrase to find, for example “hey pipecat” or “小助手”.

Returns:

The compiled pattern.