fix(proxy): guard against malformed SSE chunks in Anthropic passthrough logging

[NIK-TIGER-BILL]

Problem

Closes #24281

When proxying models like anthropic/deepseek-reasoner via the LiteLLM proxy, certain SSE stream events contain JSON structures (e.g. thinking-block events) that the Anthropic chunk parser cannot handle. This produces a noisy ERROR-level log on every streaming request:

LiteLLM Proxy:ERROR: streaming_handler.py:197 - Error in _route_streaming_logging_to_handler:
Expecting value: line 1 column 3 (char 2)

Root Cause

In _build_complete_streaming_response, the per-event loop inside AnthropicPassthroughLoggingHandler only caught StopIteration / StopAsyncIteration. Any other exception (json.JSONDecodeError, KeyError, …) from convert_str_chunk_to_generic_chunk propagated all the way up to the broad except Exception in _route_streaming_logging_to_handler — which swallowed it silently but not before logging an ERROR and aborting the entire logging call for that request.

Solution

Add an except Exception guard around the per-event conversion in _build_complete_streaming_response:

• Unparseable individual events are skipped with a DEBUG-level log
• All valid events in the same stream are still processed and logged
• The overall logging call is no longer aborted by a single malformed event

except Exception as e:
    verbose_proxy_logger.debug(
        "Skipping unparseable Anthropic SSE event during passthrough "
        "logging. Error: %s. Event (truncated): %.200s",
        str(e), event_str,
    )
    continue

Testing

• Verified with a mock deepseek-reasoner response that includes thinking-block SSE events
• ERROR log no longer appears; valid chunks are still logged correctly

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 22, 2026 7:20am

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

NIK-TIGER-BILL seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing NIK-TIGER-BILL:fix/streaming-handler-malformed-chunk-guard (e00b698) with main (c89496f)}

[greptile-apps]

Greptile Summary

This PR adds a defensive except Exception guard inside the per-event loop of AnthropicPassthroughLoggingHandler._build_complete_streaming_response. Previously, any parse failure from convert_str_chunk_to_generic_chunk (e.g. a json.JSONDecodeError caused by thinking-block events from anthropic/deepseek-reasoner) propagated out of the method and was caught by the broad handler in _route_streaming_logging_to_handler, which aborted the entire logging call with an ERROR-level log. The fix skips the individual malformed event at DEBUG level and continues processing the remaining events, which is the correct behavior for a best-effort logging path.

• The fix is well-scoped and low-risk: it only affects the logging side-path, not the actual streaming response delivered to the client.
• The except Exception is intentionally broad but could silently hide unrelated programming errors (e.g. AttributeError); narrowing it to (json.JSONDecodeError, KeyError, ValueError) would be more precise.
• A pre-existing issue (not introduced by this PR) was identified: the break on StopIteration/StopAsyncIteration only exits the inner event loop, so the outer chunk loop continues processing further chunks even after the stream signals it is exhausted.
• No automated test is included; the PR description mentions manual verification only.

Confidence Score: 4/5

• Safe to merge with minor caveats — the change is confined to a logging side-path with no impact on live traffic.
• The change is a single, targeted guard in a best-effort logging function. It correctly demotes noisy ERROR-level noise to DEBUG and keeps valid events flowing. The broad except Exception is slightly imprecise and no automated regression test is added, preventing a 5, but neither issue blocks correctness in the real request path.
• No files require special attention beyond the noted exception-type precision and the pre-existing StopIteration scoping issue in anthropic_passthrough_logging_handler.py.

Important Files Changed

Filename	Overview
litellm/proxy/pass_through_endpoints/llm_provider_handlers/anthropic_passthrough_logging_handler.py	Adds a broad except Exception guard in _build_complete_streaming_response to skip unparseable SSE events at DEBUG level rather than propagating errors through the logging stack; logical fix but no automated test is included.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[all_chunks loop] --> B[_split_sse_chunk_into_events]
    B --> C[individual_events loop]
    C --> D[convert_str_chunk_to_generic_chunk]
    D -->|Success| E[append to all_openai_chunks]
    E --> C
    D -->|StopIteration / StopAsyncIteration| F[break inner loop]
    F --> A
    D -->|Exception - pre-fix| G[propagates to _route_streaming_logging_to_handler]
    G --> H[ERROR log + logging call aborted]
    D -->|Exception - post-fix NEW| I[DEBUG log, skip event]
    I --> C
    A -->|done| J[stream_chunk_builder all_openai_chunks]
    J --> K[complete_streaming_response]

Loading

_{Last reviewed commit: "fix(proxy): guard ag..."}

[greptile-apps]

Consider narrowing the exception type

The bare except Exception will silently swallow any unexpected error that occurs inside convert_str_chunk_to_generic_chunk — including AttributeError, TypeError, ValueError, and similar programming errors that are unrelated to malformed SSE data. For a logging path this is low-risk, but it does make future bugs harder to detect.

The PR description specifically calls out json.JSONDecodeError and KeyError as the known failure modes. Catching only those (plus perhaps ValueError for other parse errors) would let genuine coding mistakes surface while still suppressing the known noisy failures:

except (json.JSONDecodeError, KeyError, ValueError) as e:
    verbose_proxy_logger.debug(
        "Skipping unparseable Anthropic SSE event during passthrough "
        "logging. Error: %s. Event (truncated): %.200s",
        str(e),
        event_str,
    )
    continue

If the intent is truly "skip everything no matter what," a short inline comment explaining that reasoning would help future readers understand it's intentional.

[greptile-apps]

break only exits the inner loop

When StopIteration or StopAsyncIteration is raised by convert_str_chunk_to_generic_chunk, the existing break only exits the inner for event_str in individual_events: loop. The outer for _chunk_str in all_chunks: loop then proceeds to the next chunk, which is unlikely to be correct — a StopIteration from the iterator signals that the stream is done, so processing should stop entirely.

This is pre-existing behavior (not introduced by this PR), but the new except Exception block directly follows it and the addition of continue makes the control-flow choices here more visible. It would be worth auditing whether the outer loop should also be exited when StopIteration is caught, for example:

except (StopIteration, StopAsyncIteration):
    break  # exits inner loop; consider `goto` outer loop or a flag

A simple fix with a sentinel flag:

stream_exhausted = False
for _chunk_str in all_chunks:
    if stream_exhausted:
        break
    individual_events = ...
    for event_str in individual_events:
        try:
            ...
        except (StopIteration, StopAsyncIteration):
            stream_exhausted = True
            break
        except Exception as e:
            ...
            continue