Skip to content

voice: strip markdown emphasis in CJK text - #2175

Open
rosetta-livekit-bot[bot] wants to merge 3 commits into
mainfrom
leads-packed-earnests
Open

voice: strip markdown emphasis in CJK text#2175
rosetta-livekit-bot[bot] wants to merge 3 commits into
mainfrom
leads-packed-earnests

Conversation

@rosetta-livekit-bot

@rosetta-livekit-bot rosetta-livekit-bot Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Ports livekit/agents#6560 to the Node.js voice text transform.

  • strip 1-3 asterisk emphasis delimiters in CJK, kana, Thai, and Korean text
  • preserve CommonMark intraword underscore behavior and non-markdown operators/identifiers
  • remove complete horizontal-rule lines without breaking streamed prefixes
  • skip inline processing when no markdown marker is present
Source diff coverage

Source diff coverage

  • Adapted: livekit-agents/livekit/agents/voice/transcription/filters.py -> agents/src/voice/transcription/text_transforms.ts. Ported all emphasis, horizontal-rule, incomplete-pattern, streaming, and fast-path behavior. Python's Unicode-aware \w boundaries are represented with JavaScript Unicode property escapes because JavaScript \w is ASCII-only.
  • Adapted: tests/test_transcription_filter.py -> agents/src/voice/transcription/text_transforms.test.ts. Ported every emphasis, preservation, horizontal-rule, and chunk-independence case added or modified by the source PR into the target's Vitest table style; retained unrelated target-only transform tests.
  • Not applicable: none. The target already had the required streaming text-transform infrastructure.

Validation

  • pnpm test agents (1,836 passed, 5 skipped)
  • pnpm build (40 packages)
  • pnpm lint (passes with pre-existing warnings)
  • pnpm format:check
  • cue-cli voice dispatch: raw assistant text 这是**很重要**的文本。; TTS usage reported 9 characters, matching transformed 这是很重要的文本。

Source: livekit/agents#6560


Ported from livekit/agents#6560

Original PR description
  • Strip markdown emphasis where the delimiter sits flush against a word character — CJK, kana and Thai are written without spaces, and Korean attaches its particles directly after the closing run. 这是**很重要**的文本, テスト**強調**です and 한국어**강조**입니다 all previously reached TTS with the asterisks spoken aloud. Underscores keep the full word boundary, so snake_case and __dunder__ stay intact.
  • Handle ***bold italic***, collapsing the three delimiter widths into one pattern.
  • Drop horizontal rules (--- / *** / ___), which carry no spoken content. Following CommonMark, the markers may be spaced apart (* * *) and a fourth column of indent makes the line code rather than a rule. Rules are stripped before the line patterns, which would otherwise read a spaced rule's first marker as a list item.
  • Require emphasis delimiters to hug their body on both sides, for underscores as well as asterisks, so a _ b _ c is left alone.
  • Skip the inline pass for text containing no markup characters.

@rosetta-livekit-bot
rosetta-livekit-bot Bot requested a review from a team as a code owner July 30, 2026 14:28
@changeset-bot

changeset-bot Bot commented Jul 30, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 82a851f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages
Name Type
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-anthropic Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-azure Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-did Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-fishaudio Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-hume Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-krisp Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-liveavatar Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-minimax Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-mistralai Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-perplexity Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-protoface Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugin-soniox Patch
@livekit/agents-plugin-tavus Patch
@livekit/agents-plugins-test Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugin-xai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@rosetta-livekit-bot
rosetta-livekit-bot Bot requested a review from longcw July 30, 2026 14:28

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

u9g added 2 commits July 30, 2026 10:53
unbalanced() built two RegExp objects per call on the streaming path and pushed
escaping onto callers, which passed a pre-escaped '\\*'. The delimiters are
literals, so splitting on them counts the same and keeps the call sites plain.
Mirrors the source PR's shape (livekit/agents#6560), which collapsed the two
per-delimiter blocks into one condition.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant