Skip to content

SSE streaming inspection filter #89

Description

@shaneutt

Parse Server-Sent Events from inference responses, exposing per-event hooks for inspection and modification by downstream filters.

Requirements

  • SSE event parser for chunked transfer-encoded responses
  • Per-event filter hooks: inspect, modify, or suppress individual events
  • Support for OpenAI-style data: {...} events and data: [DONE] sentinel
  • Streaming token extraction from delta content fields
  • Event buffering for cross-chunk event reassembly
  • Backpressure handling: don't buffer unbounded events

Acceptance Criteria

  • Integration test: SSE stream parsed, per-event hook fires for each event
  • Token count extracted from streaming deltas matches non-streaming usage response
  • Malformed SSE events handled gracefully (logged, forwarded as-is)
  • No measurable latency added for pass-through (no-op hook)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions