fix(presidio): handle Anthropic native SSE bytes in streaming unmask hook

[firestaerter3]

Problem

Two related issues prevent PII unmasking from working on the Anthropic native
streaming path:

1. Wrong chunk type in streaming hook

async_post_call_streaming_iterator_hook assembles chunks with
stream_chunk_builder(), which expects ModelResponseStream objects. With the
Anthropic native API path, chunks are raw SSE bytes. The hook silently
yields them unprocessed — PII tokens in the response text are never restored.

2. pii_tokens discarded in non-streaming output handler

AnthropicMessagesHandler.process_output_response() builds a fresh local
request_data = {"response": response}, discarding any pii_tokens the
unified guardrail forwarded from the input masking phase (via #22879).

Part of #22821.

Approach and trade-off

PII unmasking requires the complete response text before tokens can be
restored — token strings may span multiple SSE delta events. This
implementation buffers all chunks, detects the format (bytes vs
ModelResponseStream), and for Anthropic native SSE:

1. Parses SSE events (split on \n\n)
2. Collects all text_delta texts
3. Concatenates and runs _unmask_pii_text() in one pass
4. Emits non-text events unchanged, followed by a single content_block_delta
   containing the fully unmasked text

Known trade-off: when output_parse_pii is enabled, progressive streaming
output is replaced by a single delivery at response completion. This is
unavoidable for correct PII token restoration.

Changes

presidio.py — streaming hook (Patch 3)

• Collect bytes chunks alongside ModelResponseStream in the accumulation loop
• Add Anthropic native SSE branch: buffer → parse → unmask → rebuild → yield
• Extend the exception fallback to replay buffered bytes chunks

anthropic/chat/guardrail_translation/handler.py (Patches 6a + 6b)

• Accept request_data: Optional[dict] = None in process_output_response()
• Merge caller's dict: request_data = {**(request_data or {}), "response": response}
  so pii_tokens forwarded by the unified guardrail are preserved

Dependency

Requires #22879 (fix/guardrail-request-data-passthrough) for pii_tokens
plumbing through the unified guardrail layer.

Testing

tests/guardrails_tests/test_presidio_sse_unmask.py — four tests:

• process_output_response() merges caller request_data including pii_tokens
• Streaming hook unmasks PII in Anthropic native SSE bytes
• Pass-through unchanged when output_parse_pii=False
• Tokens split across multiple delta events are correctly reassembled

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Error		Mar 7, 2026 11:29am

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

Roland seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR fixes two related issues blocking PII unmasking on the Anthropic native streaming path: (1) the streaming hook now detects raw bytes chunks from Anthropic native SSE and processes them in a dedicated parse→collect→unmask→rebuild pipeline rather than silently skipping them, and (2) AnthropicMessagesHandler.process_output_response() (and all sibling handlers across 13 files) now accept a request_data parameter that is merged into effective_request_data, preserving pii_tokens forwarded from the input masking phase. The UnifiedLLMGuardrails layer was updated to thread the full request data dict through both process_output_response() and process_output_streaming_response() calls.

Key observations:

• The rebuilt text_delta_event in the SSE bytes path is constructed with a trailing \n that, when \n\n is appended at yield time, produces three consecutive newlines — a malformed SSE event separator that violates the spec (see inline comment).
• The apply_to_output=True masking path for bytes-only Anthropic native SSE now passes the stream through unmodified with a WARNING log, silently skipping output masking. This is better than the previous behavior (which would return an empty stream) but is not documented in the PR description as a behavior change; users with apply_to_output=True on Anthropic native paths will receive unmasked output without a clear error.
• The documented trade-off (progressive streaming is replaced by a single delivery when output_parse_pii=True) is correctly noted in code comments.
• All new tests are mock-only with no real network calls, consistent with project conventions.

Confidence Score: 3/5

• Mostly safe but contains a malformed SSE output issue in the rebuilt text_delta event that could affect strict SSE clients.
• The request_data passthrough across all handlers is a clean, low-risk change. The Anthropic native SSE unmasking logic is well-structured and handles edge cases (split tokens, multiple content blocks, mixed streams). However, the text_delta_event trailing \n produces three newlines when the \n\n event terminator is appended, violating the SSE protocol for any consumer that processes the rebuilt stream. Additionally, the apply_to_output=True masking path silently skips PII masking for Anthropic native bytes streams — an undocumented behavior change that could surprise users.
• litellm/proxy/guardrails/guardrail_hooks/presidio.py — the SSE event reconstruction logic around lines 1229–1232 needs the trailing \n removed from text_delta_event.

Important Files Changed

Filename	Overview
litellm/proxy/guardrails/guardrail_hooks/presidio.py	Core streaming hook change: adds Anthropic native SSE bytes handling for both masking and unmasking paths. The unmasking path correctly parses SSE events, reassembles text deltas, calls _unmask_pii_text, and rebuilds the stream in original order — but the rebuilt text_delta_event includes a trailing \n that produces three newlines when the \n\n event terminator is appended, violating the SSE spec. The apply_to_output=True bytes-only path silently skips masking (only a WARNING is logged), which is a behavior change not covered by the PR description.
litellm/llms/anthropic/chat/guardrail_translation/handler.py	Adds request_data: Optional[dict] = None parameter to process_output_response() and merges it into effective_request_data so pii_tokens forwarded from the input masking phase are preserved and passed to apply_guardrail().
litellm/llms/base_llm/guardrail_translation/base_translation.py	Abstract base updated to declare request_data: Optional[dict] = None on both process_output_response() and process_output_streaming_response(), keeping the interface in sync with all implementing subclasses.
litellm/proxy/guardrails/guardrail_hooks/unified_guardrail/unified_guardrail.py	Threads the full request_data (or data) dict through to each process_output_response() and process_output_streaming_response() call so downstream handlers receive pii_tokens without any other side effects.
litellm/llms/openai/chat/guardrail_translation/handler.py	Mirrors the Anthropic handler change: adds request_data parameter, renames local dict to effective_request_data, and merges caller data before passing to apply_guardrail(). Both the non-streaming and streaming output paths are updated.
tests/guardrails_tests/test_presidio_sse_unmask.py	Four new mock-only unit tests covering: (1) process_output_response() preserves pii_tokens from caller request_data, (2) streaming hook unmasks PII in Anthropic native SSE bytes, (3) pass-through when output_parse_pii=False, (4) token reassembly across split delta events. All tests use mocks with no real network calls, satisfying the project rule. Uses __new__ to bypass __init__ which is fragile but works given the limited attribute access in the tested paths.
tests/guardrails_tests/test_guardrail_request_data_passthrough.py	Two introspection-based tests verifying that BaseTranslation and a representative set of concrete handlers all declare the request_data parameter with a None default, guarding against signature drift.

Sequence Diagram

sequenceDiagram
    participant Client
    participant UnifiedGuardrail
    participant PresidioHook
    participant AnthropicHandler
    participant Anthropic

    Client->>UnifiedGuardrail: async_post_call_streaming_iterator_hook(request_data)
    UnifiedGuardrail->>PresidioHook: async_post_call_streaming_iterator_hook(response, request_data)
    Note over PresidioHook: output_parse_pii=True, pii_tokens in request_data

    PresidioHook->>PresidioHook: collect bytes chunks (Anthropic native SSE)
    Note over PresidioHook: remaining_bytes_chunks populated
    PresidioHook->>PresidioHook: split on \\n\\n → raw_events
    PresidioHook->>PresidioHook: classify events (text_delta vs non-text)
    PresidioHook->>PresidioHook: _unmask_pii_text(combined_text, pii_tokens)
    PresidioHook->>PresidioHook: rebuild merged event at original slot position
    PresidioHook-->>Client: yield event + b"\\n\\n" (per event)

    Client->>UnifiedGuardrail: async_post_call_success_hook(response, request_data)
    UnifiedGuardrail->>AnthropicHandler: process_output_response(response, guardrail, request_data)
    Note over AnthropicHandler: effective_request_data = {**request_data, "response": response}
    AnthropicHandler->>PresidioHook: apply_guardrail(inputs, effective_request_data)
    PresidioHook-->>AnthropicHandler: guardrailed_inputs (pii_tokens preserved)
    AnthropicHandler-->>UnifiedGuardrail: updated response

Loading

Comments Outside Diff (1)

1. litellm/proxy/guardrails/guardrail_hooks/presidio.py, line 1229-1232 (link)

   Malformed SSE output — extra trailing newline in rebuilt text_delta_event

   text_delta_event is constructed with a trailing \n after the data: line. When it is later yielded via yield event + b"\n\n", the resulting byte stream has three consecutive newlines (\n\n\n) instead of the required two (\n\n) that terminate an SSE event.

   Non-text events don't have this problem because they arrive from combined.split(b"\n\n") and therefore carry no trailing newline before the \n\n is appended.

   A strict SSE parser will correctly terminate the event at the first blank line, but will then see an extra empty "event" from the residual third newline. While most consumers silently discard empty events, this is a protocol violation that can trip up spec-compliant clients.

_{Last reviewed commit: 8b1eb08}

[greptile-apps]

Broken SSE event ordering — protocol events emitted before text content

The rebuilt SSE byte stream is assembled as non_text_events + [text_delta_event]. However, a real Anthropic SSE stream delivers lifecycle events that come after text deltas (i.e., content_block_stop, message_delta, message_stop) in that same non_text_events list. Placing them all before the new text_delta_event breaks the Anthropic SSE protocol order for any consumer that follows the spec.

Example: for a normal stream content_block_start → content_block_delta → content_block_stop → message_delta → message_stop, the rebuilt stream becomes:

content_block_start   ← correct
content_block_stop    ← wrong: comes before the text delta
message_delta         ← wrong: comes before the text delta
message_stop          ← wrong: comes before the text delta
content_block_delta   ← actual text, after "stop" events

A correct approach would preserve the original stream order, inserting the merged unmasked content_block_delta in the position of the first text_delta event, and dropping subsequent text_delta events:

# Instead of splitting into text/non-text buckets, preserve event order
rebuilt_events: List[bytes] = []
text_injected = False
for raw_event in raw_events:
    # ... parse as before ...
    if is_text_delta:
        if not text_injected:
            rebuilt_events.append(merged_text_delta_event)
            text_injected = True
        # else: skip additional text_delta events (merged above)
    else:
        rebuilt_events.append(raw_event)

[greptile-apps]

Hardcoded index: 0 loses the original content block index

The rebuilt text_delta_payload always uses "index": 0. In responses that contain multiple content blocks (e.g., extended thinking at index 0 and actual text at index 1), all collected text parts are re-emitted at index 0, misreporting the block identity.

The correct index should be tracked from the original text_delta events — all text deltas are expected to share the same block index, so the index from the first text_delta event can be reused:

# Track the original index of the text content block
text_block_index = 0  # default fallback

for raw_event in raw_events:
    # ... parse as before ...
    if is_text_delta:
        text_block_index = payload.get("index", 0)  # capture once
        text_parts.append(delta.get("text", ""))

text_delta_payload = {
    "type": "content_block_delta",
    "index": text_block_index,   # use original index
    "delta": {"type": "text_delta", "text": unmasked_text},
}

[greptile-apps]

Mixed-type chunks — bytes chunks are silently dropped

The branch condition if remaining_bytes_chunks and not remaining_chunks: only enters the Anthropic native SSE path when all chunks are bytes. If a stream happens to contain both ModelResponseStream and bytes chunks (e.g., a wrapped or middleware-modified stream), the bytes chunks are silently discarded — they are collected into remaining_bytes_chunks (line 1119) but are never yielded or processed, because the code falls through to the ModelResponseStream path.

The exception handler (lines 1223–1224) does yield both types, confirming mixed streams were considered possible. The normal path should also handle them:

if remaining_bytes_chunks and not remaining_chunks:
    # ... Anthropic native SSE path ...
elif remaining_bytes_chunks and remaining_chunks:
    # Mixed stream — unexpected, log and pass bytes through
    verbose_proxy_logger.warning(
        "Unexpected mixed bytes+ModelResponseStream chunks; bytes chunks passed through unmodified"
    )
    for chunk in remaining_bytes_chunks:
        yield chunk  # type: ignore[misc]
    # Continue to handle ModelResponseStream chunks below

[firestaerter3]

Superseded by #23037 — a single clean PR covering the full Presidio PII round-trip fix end-to-end (position bug, cross-instance token loss, apply_guardrail unmask path, Anthropic native dict + SSE streaming). Closing in favour of that PR.