client(JournalClient.list): consume the server-streaming List response#51
Open
jshearer wants to merge 1 commit into
Open
client(JournalClient.list): consume the server-streaming List response#51jshearer wants to merge 1 commit into
jshearer wants to merge 1 commit into
Conversation
The broker's List RPC is server-streaming and caps the number of journals per `ListResponse` message, so a collection with more journals than that cap returns its listing as several newline-delimited `{result}` frames. `list()` called `response.json()` once, which parses only the first frame and drops every journal after it (and throws outright for callers that read the whole body).
* Pipe the response through the same `splitStream`/`parseJSONStream`/`unwrapResult` stages `read()` uses, drain it, and merge each frame's `journals`, so listings of any size return completely.
* Return the full `ProtocolListResponse` rather than `ProtocolJournalSpec[]`, preserving `modRevision`/`route` and matching the streamed shape.
* Switch the error path to `ResponseError.fromStreamResponse`, since the streaming endpoint returns the wrapped `{error}` envelope instead of the unary `{code}` shape.
jshearer
added a commit
to estuary/ui
that referenced
this pull request
Jun 23, 2026
…urnals The broker's journal `List` RPC is server-streaming and caps each `ListResponse` frame at 25 journals (gazette/core#466, down from 1000), so a collection split more than 25 ways returns its listing as several newline-delimited frames. `getJournals` parsed the body with a single `response.json()`, which failed on the second frame and surfaced as the generic "Server Error", leaving the preview with no journal to read. * Bump vendored `data-plane-gateway` to a build whose `JournalClient.list()` consumes the stream and merges every frame (estuary/data-plane-gateway#51). * Call `journalClient.list()` directly and remove the now-dead `getJournals` and `isNestedProtocolListResponse` helpers.
jshearer
added a commit
to estuary/ui
that referenced
this pull request
Jun 23, 2026
…urnals The broker's journal `List` RPC is server-streaming and caps each `ListResponse` frame at 25 journals (gazette/core#466, down from 1000), so a collection split more than 25 ways returns its listing as several newline-delimited frames. `getJournals` parsed the body with a single `response.json()`, which failed on the second frame and surfaced as the generic "Server Error", leaving the preview with no journal to read. * Bump vendored `data-plane-gateway` to a build whose `JournalClient.list()` consumes the stream and merges every frame (estuary/data-plane-gateway#51). * Call `journalClient.list()` directly and remove the now-dead `getJournals` and `isNestedProtocolListResponse` helpers.
jshearer
added a commit
to estuary/ui
that referenced
this pull request
Jun 23, 2026
…urnals The broker's journal `List` RPC is server-streaming and caps each `ListResponse` frame at 25 journals (gazette/core#466, down from 1000), so a collection split more than 25 ways returns its listing as several newline-delimited frames. `getJournals` parsed the body with a single `response.json()`, which failed on the second frame and surfaced as the generic "Server Error", leaving the preview with no journal to read. * Bump vendored `data-plane-gateway` to a build whose `JournalClient.list()` consumes the stream and merges every frame (estuary/data-plane-gateway#51). * Call `journalClient.list()` directly and remove the now-dead `getJournals` and `isNestedProtocolListResponse` helpers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The broker's List RPC is server-streaming and caps the number of journals per
ListResponsemessage, so a collection with more journals than that cap returns its listing as several newline-delimited{result}frames.list()calledresponse.json()once, which parses only the first frame and drops every journal after it (and throws outright for callers that read the whole body).splitStream/parseJSONStream/unwrapResultstagesread()uses, drain it, and merge each frame'sjournals, so listings of any size return completely.ProtocolListResponserather thanProtocolJournalSpec[], preservingmodRevision/routeand matching the streamed shape.ResponseError.fromStreamResponse, since the streaming endpoint returns the wrapped{error}envelope instead of the unary{code}shape.