Skip to content

fix: check [DONE] SSE marker before deserialization in stream_mapped_raw_events#543

Merged
64bit merged 1 commit into
64bit:mainfrom
Linuxdazhao:fix/done-sse-deserialization
May 11, 2026
Merged

fix: check [DONE] SSE marker before deserialization in stream_mapped_raw_events#543
64bit merged 1 commit into
64bit:mainfrom
Linuxdazhao:fix/done-sse-deserialization

Conversation

@Linuxdazhao
Copy link
Copy Markdown
Contributor

Summary

Fixes #542[DONE] SSE end-of-stream marker being passed to serde_json::from_str in stream(), causing a deserialization error.

Root Cause

In v0.36.0 (PR #537), the streaming pipeline was refactored to remove reqwest-eventsource. The old stream() function checked for [DONE] before deserialization. The new stream() delegates to stream_mapped_raw_events, which checks [DONE] after the mapper closure — and the mapper in stream() is serde_json::from_str.

Fix

Move the [DONE] check before the event_mapper call in both implementations:

  • WASM (futures::stream::unfold): Check [DONE] and return None to end the stream cleanly. Removed the finished state variable since it's no longer needed.
  • Non-WASM (tokio::mpsc): Check [DONE] and break before calling event_mapper.

Test Plan

  • Existing unit tests pass (15 passed, 11 skipped — require API key)
  • Change is minimal and restores the pre-refactor behavior

Before

let done = event.data == "[DONE]";
// ...
let response = event_mapper(event); // [DONE] passed to JSON parser → error!
return Some((response, (event_stream, event_mapper, done)));

After

if event.data == "[DONE]" {
    return None; // clean stream termination
}
// ...
let response = event_mapper(event);
return Some((response, (event_stream, event_mapper)));

…raw_events

PR #537 refactored the streaming pipeline and removed reqwest-eventsource.
The old `stream()` function checked for the `[DONE]` SSE end-of-stream
marker *before* passing event data to `serde_json::from_str`. The
refactored version delegated to `stream_mapped_raw_events`, which checks
`[DONE]` *after* calling the `event_mapper` closure. Since the mapper in
`stream()` deserializes JSON, this causes `[DONE]` to fail with:

  failed deserialization of: [DONE]

Move the `[DONE]` check before the mapper call in both the WASM
(futures::stream::unfold) and non-WASM (tokio::mpsc) implementations.

Closes #542
@64bit 64bit merged commit 4b51dc6 into 64bit:main May 11, 2026
99 checks passed
@64bit
Copy link
Copy Markdown
Owner

64bit commented May 11, 2026

I was trying to reduce duplicate functions - good catch and thanks for the fix!

@Linuxdazhao Linuxdazhao deleted the fix/done-sse-deserialization branch May 12, 2026 02:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: [DONE] SSE marker fails deserialization in stream() after v0.36.0 refactor

2 participants