Skip to content

fix(core): properly propagate stream cancellation on disconnect (#1349) #1354

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Sigmabrogz wants to merge 4 commits into
base: main
Choose a base branch
from
+64 −306
Open

fix(core): properly propagate stream cancellation on disconnect (#1349) #1354

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2ac57d3
fix(core): propagate stream cancellation to avoid leaking listeners
Sigmabrogz Mar 12, 2026
8b9ba82
Merge branch 'main' into fix-1349-stream-cancel
Sigmabrogz Mar 12, 2026
52c3fd0
test(core): add flushablePipe abort test
Mar 14, 2026
3b0cf54
test(core): fix unit test assertions for flushablePipe
Mar 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/weak-beds-cry.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@workflow/core": patch
---

fix(core): properly propagate stream cancellation on disconnect
39 changes: 39 additions & 0 deletions packages/core/src/flushable-stream.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,45 @@
expect(streamClosed).toBe(true);
});

it('should handle abort signal propagating back up flushablePipe when reader disconnects early', async () => {
const chunks: string[] = [];
let sinkAborted = false;

// Create a sink that aborts (representing a dropped connection)
const mockSink = new WritableStream<string>({
write(chunk) {
chunks.push(chunk);
},
abort(reason) {
sinkAborted = true;
},
});

const { readable, writable } = new TransformStream<string, string>();
const state = createFlushableState();

// Start piping in background
const pipePromise = flushablePipe(readable, mockSink, state);

pollWritableLock(writable, state);

const userWriter = writable.getWriter();
await userWriter.write('valid chunk');

// Simulate a stream error / drop on the readable side (which aborts the pipe)
const error = new Error('Client disconnected');
readable.cancel(error);

Check failure on line 139 in packages/core/src/flushable-stream.test.ts

View workflow job for this annotation

GitHub Actions / Unit Tests (ubuntu-latest)

Unhandled error 

TypeError: Invalid state: ReadableStream is locked
 ❯ src/flushable-stream.test.ts:139:14

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { code: 'ERR_INVALID_STATE' }
This error originated in "src/flushable-stream.test.ts" test file. It doesn't mean the error was thrown inside the file itself, but while it was running.
The latest test that might've caused the error is "should test with pollReadableLock". It might mean one of the following:
- The error was thrown, while Vitest was running this test.
- If the error occurred after the test had been completed, this was the last documented test before it was thrown.

Check failure on line 139 in packages/core/src/flushable-stream.test.ts

View workflow job for this annotation

GitHub Actions / Unit Tests (windows-latest)

Unhandled error 

TypeError: Invalid state: ReadableStream is locked
 ❯ src/flushable-stream.test.ts:139:14

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { code: 'ERR_INVALID_STATE' }
This error originated in "src/flushable-stream.test.ts" test file. It doesn't mean the error was thrown inside the file itself, but while it was running.
The latest test that might've caused the error is "should test with pollReadableLock". It might mean one of the following:
- The error was thrown, while Vitest was running this test.
- If the error occurred after the test had been completed, this was the last documented test before it was thrown.
Copy link
Member

@VaguelySerious VaguelySerious Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const chunks: string[] = [];
let sinkAborted = false;
// Create a sink that aborts (representing a dropped connection)
const mockSink = new WritableStream<string>({
write(chunk) {
chunks.push(chunk);
},
abort(reason) {
sinkAborted = true;
},
});
const { readable, writable } = new TransformStream<string, string>();
const state = createFlushableState();
// Start piping in background
const pipePromise = flushablePipe(readable, mockSink, state);
pollWritableLock(writable, state);
const userWriter = writable.getWriter();
await userWriter.write('valid chunk');
// Simulate a stream error / drop on the readable side (which aborts the pipe)
const error = new Error('Client disconnected');
readable.cancel(error);
const chunks: string[] = [];
let sinkAborted = false;
// Create a sink that tracks writes and aborts (representing the response stream)
const mockSink = new WritableStream<string>({
write(chunk) {
chunks.push(chunk);
},
});
// Use a custom ReadableStream with a controller so we can error it
// externally. This simulates the source stream breaking (e.g., a client
// disconnect that causes the readable side of the pipe to error).
// Note: We cannot call readable.cancel() on a locked ReadableStream
// (flushablePipe locks it via getReader()), so we use controller.error()
// which propagates through the internal reader.
let sourceController!: ReadableStreamDefaultController<string>;
const source = new ReadableStream<string>({
start(controller) {
sourceController = controller;
},
});
const state = createFlushableState();
// Start piping in background
const pipePromise = flushablePipe(source, mockSink, state).catch(() => {
// Errors handled via state.reject
});
// Enqueue a valid chunk through the source
sourceController.enqueue('valid chunk');
// Allow the pipe to process the chunk
await new Promise((r) => setTimeout(r, 50));
// Simulate a stream error / client disconnect on the source side.
// controller.error() propagates to the internal reader held by flushablePipe,
// causing reader.read() to reject, which triggers the catch block.
sourceController.error(new Error('Client disconnected'));


Copy link
Contributor

@vercel vercel bot Mar 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test calls readable.cancel(error) on a ReadableStream that is locked by flushablePipe, causing the cancellation to throw rather than test the intended disconnect behavior. Additionally, pipePromise is not awaited or caught, risking unhandled promise rejections.

Fix on Vercel

// Write should fail because the underlying pipe broke
await expect(userWriter.write('another')).rejects.toThrow();

Check failure on line 142 in packages/core/src/flushable-stream.test.ts

View workflow job for this annotation

GitHub Actions / Unit Tests (ubuntu-latest)

src/flushable-stream.test.ts > flushable stream behavior > should handle abort signal propagating back up flushablePipe when reader disconnects early 

AssertionError: promise resolved "undefined" instead of rejecting

- Expected: 
Error {
  "message": "rejected promise",
}

+ Received: 
undefined

 ❯ src/flushable-stream.test.ts:142:45

Check failure on line 142 in packages/core/src/flushable-stream.test.ts

View workflow job for this annotation

GitHub Actions / Unit Tests (windows-latest)

src/flushable-stream.test.ts > flushable stream behavior > should handle abort signal propagating back up flushablePipe when reader disconnects early 

AssertionError: promise resolved "undefined" instead of rejecting

- Expected: 
Error {
  "message": "rejected promise",
}

+ Received: 
undefined

 ❯ src/flushable-stream.test.ts:142:45
Copy link
Member

@VaguelySerious VaguelySerious Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Write should fail because the underlying pipe broke
await expect(userWriter.write('another')).rejects.toThrow();
// Wait for the pipe to process the error
await pipePromise;
// State promise should reject with the disconnection error
await expect(state.promise).rejects.toThrow('Client disconnected');


// State promise should reject with the cancellation error
await expect(state.promise).rejects.toThrow('Client disconnected');

// Ensure the sink received the abort signal
expect(sinkAborted).toBe(true);
Copy link
Member

@VaguelySerious VaguelySerious Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// State promise should reject with the cancellation error
await expect(state.promise).rejects.toThrow('Client disconnected');
// Ensure the sink received the abort signal
expect(sinkAborted).toBe(true);
// The first chunk should have been written before the error
expect(chunks).toContain('valid chunk');
// Ensure the stream ended
expect(state.streamEnded).toBe(true);

});

it('should handle write errors during pipe operations', async () => {
const chunks: string[] = [];

Expand Down
13 changes: 12 additions & 1 deletion packages/core/src/flushable-stream.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,12 @@ export async function flushablePipe(

// Read from source - don't count as pending op since we're just waiting for data
// The important ops are writes to the sink (server)
const readResult = await reader.read();
const readResult = await Promise.race([
reader.read(),
writer.closed.then(() => {
throw new Error('Writable stream closed prematurely');
}),
]);

// Check if stream has ended (e.g., due to error in another path) before processing
if (state.streamEnded) {
Expand Down Expand Up @@ -257,8 +262,14 @@ export async function flushablePipe(
// while other callers may depend on this rejection. Some known callers
// explicitly ignore this rejection (`.catch(() => {})`) and rely solely
// on `state.reject(err)` for error handling.

// Attempt to cancel the upstream reader so the source knows it should stop generating data.
reader.cancel(err).catch(() => {});
Copy link
Member

@VaguelySerious VaguelySerious Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should all of these .catch(() => {}); statements instead log a warning? Presumably closing should work in most cases, and we'd like to know if it's not possible to close cleanly

Copy link
Member

@VaguelySerious VaguelySerious Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's log a warning for any cancel failures

throw err;
} finally {
// If we're exiting normally but the stream was externally ended before completion,
// we should cancel the reader to notify the source.
reader.cancel().catch(() => {});
reader.releaseLock();
writer.releaseLock();
}
Expand Down
6 changes: 6 additions & 0 deletions packages/core/src/serialization.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -445,6 +445,12 @@ export class WorkflowServerReadableStream extends ReadableStream<Uint8Array> {
controller.enqueue(result.value);
}
},
cancel: async (reason) => {
if (this.#reader) {
await this.#reader.cancel(reason).catch(() => {});
this.#reader = undefined;
}
},
});
}
}
Expand Down
Loading