Add streamTimeout option to WebSocketService/WebSocketClient; send close frame on inbound failure; clean up related resources #6357

sjy982 · 2025-08-17T10:33:18Z

Motivation

Instead of asking each handler to wrap the inbound stream with StreamMessage.timeout(..., UNTIL_NEXT), expose it as an option on WebSocketServiceBuilder / WebSocketClientBuilder so both server and client can enable it consistently and easily.
When the inbound completes abnormally (cancel/abort/timeout, etc.), further outbound sends are meaningless. We should immediately tidy up outbound and the stream/channel. If the connection is still valid and the termination is initiated locally, prefer sending a WebSocket close frame (with an appropriate status + reason) over a transport reset, for better reason propagation, interoperability, and observability.

Modifications

Add WebSocketServiceBuilder#streamTimeout(Duration).
- When set, wrap inbound in TimeoutStreamMessage with StreamTimeoutMode.UNTIL_NEXT inside DefaultWebSocketService#serve(...).
Update DefaultWebSocketService#serve(...):
- If inbound completes with an error, call outbound.abort(mappedCause) so recoverAndResume sends a CloseWebSocketFrame.
- Map CancelledSubscriptionException / AbortedStreamException to InboundCompleteException to avoid being skipped by the recovery logic.
Add WebSocketClientBuilder#streamTimeout(Duration).
- When set, wrap inbound in TimeoutStreamMessage with StreamTimeoutMode.UNTIL_NEXT inside DefaultWebSocketClient#connect(...).
Update WebSocketSession#setOutbound(...):
- Attach recoverAndResume(...) to outbound so that on send failure it emits a CloseWebSocketFrame and records the exception in RequestLog (endRequest(cause) / endResponse(cause)). For ClosedStreamException, do not send a close frame—just abort.
- When inbound completes exceptionally, abort(...) outbound with the mapped cause so the recovery stream kicks in.
Add InboundCompleteException extends CancellationException:
- Signals that inbound completed due to cancellation/abort.
Add a client-side newCloseWebSocketFrame(Throwable) in WebSocketUtil.

Result

WebSocketService / WebSocketClient can apply StreamMessage.timeout(..., UNTIL_NEXT) to inbound via the simple streamTimeout(Duration) option.
When inbound completes with an error:
- If the channel is still usable, abort outbound with the mapped cause so a close frame is sent (skip sending for ClosedStreamException).
- Record request/response causes in RequestLog.
- On HTTP/1.x, close the channel; on HTTP/2, clean up the stream.

…uestLog, ensure close frame on cancel/abort; add tests (H2C close-frame/log, H1 channel close)

…utbound, closed outbound when inbound ends, introduced InboundCompleteException, and added a client-side newCloseWebSocketFrame() in WebSocketUtil.

github-actions · 2025-08-17T11:19:52Z

🔍 Build Scan® (commit: `7315e52`)

Job name	Status	Build Scan®
build-ubicloud-standard-16-jdk-8	✅	https://ge.armeria.dev/s/bm43lhrrpegnw
build-ubicloud-standard-16-jdk-21-snapshot-blockhound	✅	https://ge.armeria.dev/s/wcgst5gh5tzzm
build-ubicloud-standard-16-jdk-17-min-java-17-coverage	❌ (failure)	https://ge.armeria.dev/s/4s265c27yqdgc
build-ubicloud-standard-16-jdk-17-min-java-11	✅	https://ge.armeria.dev/s/35mkbdpex4q76
build-ubicloud-standard-16-jdk-17-leak	❌ (failure)	https://ge.armeria.dev/s/vswshzqvpsino
build-ubicloud-standard-16-jdk-11	✅	https://ge.armeria.dev/s/nu3wjct2nb57c
build-macos-latest-jdk-21	✅	https://ge.armeria.dev/s/2uuqvzlyy2bia

jrhee17

Looks good overall, left some comments

core/src/main/java/com/linecorp/armeria/common/stream/TimeoutStreamMessage.java

core/src/main/java/com/linecorp/armeria/server/websocket/WebSocketServiceBuilder.java

core/src/main/java/com/linecorp/armeria/client/websocket/WebSocketSession.java

codecov · 2025-08-28T15:11:47Z

Codecov Report

❌ Patch coverage is 80.00000% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.16%. Comparing base (8150425) to head (2184b0c).
⚠️ Report is 215 commits behind head on main.

Files with missing lines	Patch %	Lines
...rp/armeria/common/stream/TimeoutStreamMessage.java	73.68%	5 Missing ⚠️
...orp/armeria/client/websocket/WebSocketSession.java	77.77%	2 Missing and 2 partials ⚠️
...corp/armeria/common/logging/DefaultRequestLog.java	82.35%	1 Missing and 2 partials ⚠️
...ecorp/armeria/common/InboundCompleteException.java	50.00%	2 Missing ⚠️
...ommon/websocket/WebSocketIdleTimeoutException.java	50.00%	2 Missing ⚠️
...meria/client/websocket/WebSocketClientBuilder.java	80.00%	0 Missing and 1 partial ⚠️
.../linecorp/armeria/common/stream/StreamMessage.java	0.00%	0 Missing and 1 partial ⚠️
...eria/server/websocket/WebSocketServiceBuilder.java	80.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #6357      +/-   ##
============================================
- Coverage     74.46%   74.16%   -0.30%     
- Complexity    22234    23035     +801     
============================================
  Files          1963     2064     +101     
  Lines         82437    86188    +3751     
  Branches      10764    11317     +553     
============================================
+ Hits          61385    63922    +2537     
- Misses        15918    16859     +941     
- Partials       5134     5407     +273

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

fix: In WebSocketSession endRequest -> requestCause

core/src/main/java/com/linecorp/armeria/client/websocket/WebSocketSession.java

core/src/main/java/com/linecorp/armeria/common/stream/TimeoutStreamMessage.java

core/src/main/java/com/linecorp/armeria/internal/common/websocket/WebSocketUtil.java

core/src/main/java/com/linecorp/armeria/internal/server/websocket/DefaultWebSocketService.java

… frame (#6375) Motivation: Currently, once an `HttpRequest` and `HttpResponse` is completed, the underlying HTTP2 stream is cancelled using a `RST_STREAM`. This makes sense for normal HTTP constructs since it indicates that we are no longer interested in the request, and we would like to release resources associated with it. However, some protocols such as WebSockets implement their own graceful shutdown procedure. In detail, Armeria's `HttpRequest`, `HttpResponse` implements WebSocket graceful shutdown and the reactive stream implementation is closed when a `CLOSE` frame is both sent and received. However, although the websocket session is completed, the underlying HTTP2 stream may not necessarily be complete. Protocol-wise, there is an inherent discrepancy between websocket session completion and HTTP2 stream completion. The current implementation defaults to sending a `RST_STREAM` immediately once the corresponding `HttpRequest` and `HttpResponse`. For websockets, I propose that a delay is given so that the remote has a chance to end the stream. Assuming that we will tie the lifecycle of inbound `WebSocket`s with outbound `WebSocket`s (so sending a close frame also closes the inbound) in #6357 , this option can be thought of similar to netty's `forceCloseTimeoutMillis` option. (which acts as a timeout since sending the CLOSE frame) Modifications: - Added a `closeHttp2StreamDelayMillis` option to `ServiceConfig` and relevant implementations - `WebSocketService` sets a `closeHttp2StreamDelayMillis` of 10 seconds by default - `HttpServerHandler` decides when to send a `RST_STREAM` based on the `closeHttp2StreamDelayMillis` - Added `Http2StreamLifecycleHandler` which maintains the lifecycle of reset futures. This ensures that scheduled futures aren't leaked for servers with high throughput. - Every time a request/response is closed but the corresponding stream is alive, `maybeResetStream` is called. - Every time a stream is closed, `notifyStreamClosed` is called to clean up possibly scheduled futures. Result: - Users can set a timeout for closing a websocket session - Fix a bug where closing a websocket session could send a `RST_STREAM` frame

core/src/main/java/com/linecorp/armeria/client/websocket/WebSocketClientBuilder.java

core/src/main/java/com/linecorp/armeria/client/websocket/WebSocketSession.java

core/src/main/java/com/linecorp/armeria/common/logging/RequestLogBuilder.java

core/src/main/java/com/linecorp/armeria/common/stream/TimeoutStreamMessage.java

core/src/main/java/com/linecorp/armeria/common/stream/StreamMessage.java

core/src/main/java/com/linecorp/armeria/internal/server/websocket/DefaultWebSocketService.java

core/src/test/java/com/linecorp/armeria/client/websocket/WebSocketClientStreamTimeoutTest.java

core/src/main/java/com/linecorp/armeria/common/InboundCompleteException.java

core/src/main/java/com/linecorp/armeria/common/stream/TimeoutStreamMessage.java

core/src/main/java/com/linecorp/armeria/common/websocket/WebSocketIdleTimeoutException.java

core/src/main/java/com/linecorp/armeria/client/websocket/WebSocketSession.java

core/src/main/java/com/linecorp/armeria/internal/server/websocket/DefaultWebSocketService.java

core/src/test/java/com/linecorp/armeria/client/websocket/WebSocketClientStreamTimeoutTest.java

ikhoon

Nice work, @sjy982! 🚀 🎉

core/src/main/java/com/linecorp/armeria/client/websocket/WebSocketClientBuilder.java

core/src/main/java/com/linecorp/armeria/client/websocket/WebSocketSession.java

core/src/main/java/com/linecorp/armeria/internal/server/websocket/DefaultWebSocketService.java

…cketClientBuilder.java Co-authored-by: Ikhun Um <[email protected]>

…cketSession.java Co-authored-by: Ikhun Um <[email protected]>

…ket/DefaultWebSocketService.java Co-authored-by: Ikhun Um <[email protected]>

core/src/main/java/com/linecorp/armeria/client/websocket/WebSocketSession.java

core/src/main/java/com/linecorp/armeria/common/InboundCompleteException.java

core/src/main/java/com/linecorp/armeria/client/websocket/WebSocketClientBuilder.java

jrhee17 · 2025-11-04T06:14:54Z

core/src/main/java/com/linecorp/armeria/client/websocket/WebSocketSession.java

+            if (cause instanceof StreamTimeoutException) {
+                wrapped = new WebSocketIdleTimeoutException("WebSocket inbound idle-timeout exceeded",
+                                                            cause);
+            } else {
+                wrapped = new InboundCompleteException("inbound stream was cancelled", cause);
+            }


Question) Is it possible to just use InboundCompleteException to indicate the outbound stream was cancelled by the inbound stream? If users would like to differentiate the reason, they could just check InboundCompleteException#cause?

Suggested change

if (cause instanceof StreamTimeoutException) {

wrapped = new WebSocketIdleTimeoutException("WebSocket inbound idle-timeout exceeded",

cause);

} else {

wrapped = new InboundCompleteException("inbound stream was cancelled", cause);

}

wrapped = new WebSocketIdleTimeoutException("WebSocket inbound idle-timeout exceeded", cause);

If we apply the code you suggested, non-timeout exceptions would also be wrapped with WebSocketIdleTimeoutException.
Or were you suggesting that we should instead use only InboundCompleteException as the single wrapper and let callers distinguish the concrete exception via its cause?

Or were you suggesting that we should instead use only InboundCompleteException as the single wrapper and let callers distinguish the concrete exception via its cause?

Right, I understood that the meaning of InboundCompleteException is that "the outbound was cancelled due to the inbound".

The reason we use WebSocketIdleTimeoutException is that the existing StreamTimeoutException may not clearly convey its meaning to WebSocket users. That’s why it was introduced, and since it already makes the intention explicit, we didn’t wrap it again with InboundCompleteException.

Do you think it would be better to wrap it once more using InboundCompleteException and include it as the cause?

I realized that I copy-pasted the incorrect exception in the suggestion above, I meant to modify the above block to:

wrapped = new InboundCompleteException("inbound stream was cancelled", cause);

because WebSocketIdleTimeoutException seems like a very specific exception that likely won't be used anywhere else - less exception types users have to handle/worry about

If you prefer to introduce the above exception, feel free to leave as-is as this is a nit comment.

Let me elaborate on why WebSocketIdleTimeoutException was added.
Many WebSocket implementations disable request timeouts and instead detect and close idle sessions. Since this behavior is quite common for users working with WebSockets, I thought it would be better to introduce a more specific error type to convey the state clearly.

core/src/main/java/com/linecorp/armeria/common/websocket/WebSocketIdleTimeoutException.java

…Exception.java Co-authored-by: jrhee17 <[email protected]>

…cketIdleTimeoutException.java Co-authored-by: jrhee17 <[email protected]>

sjy982 added 7 commits August 11, 2025 18:09

WebsocketService: apply inbound streamTimeout, record timeouts in Req…

223ccc9

…uestLog, ensure close frame on cancel/abort; add tests (H2C close-frame/log, H1 channel close)

TimeoutSubscriber#cancel() -> add completeExceptionally

ea06e02

Added streamTimeout to WebSocketClient, applied recoverAndResume to o…

e676dd7

…utbound, closed outbound when inbound ends, introduced InboundCompleteException, and added a client-side newCloseWebSocketFrame() in WebSocketUtil.

added test class

c0f516f

lint

205d335

lint

74ce3c0

handel -> exceptionally

5462bed

sjy982 requested review from ikhoon, jrhee17, minwoox and trustin as code owners August 17, 2025 10:33

jrhee17 reviewed Aug 20, 2025

View reviewed changes

sjy982 requested a review from jrhee17 August 25, 2025 06:21

jrhee17 mentioned this pull request Aug 28, 2025

Fix a bug where closing a websocket session could send a RST_STREAM frame #6375

Merged

fix

1e0510c

feat: Add requestCause(throwable) to RequestLog,

8a599ac

fix: In WebSocketSession endRequest -> requestCause

ikhoon added the new feature label Sep 2, 2025

ikhoon reviewed Sep 2, 2025

View reviewed changes

sjy982 added 2 commits September 2, 2025 14:38

fix: TimeoutStreamMessage.class -> change name

58e8ae7

fix: mapped -> wrapped

e91bb28

sjy982 requested a review from ikhoon September 2, 2025 10:47

fix: recoverAndResume -> endWith

69a35cc

fix: CompletableFuture -> abort(ex)

1de6137

github-actions bot added the Stale label Oct 21, 2025

Merge branch 'main' into feat/ws-timeout-support

542a8f0

ikhoon removed the Stale label Oct 24, 2025

ikhoon reviewed Oct 24, 2025

View reviewed changes

sjy982 added 2 commits October 28, 2025 16:49

fix: Apply review

1a7facc

lint

ae04fd8

sjy982 requested a review from ikhoon October 28, 2025 07:51

Merge branch 'main' into feat/ws-timeout-support

ab2729a

ikhoon reviewed Oct 30, 2025

View reviewed changes

ikhoon added this to the 1.34.0 milestone Oct 30, 2025

fix: Apply review

2184b0c

sjy982 requested a review from ikhoon October 31, 2025 09:00

ikhoon approved these changes Nov 3, 2025

View reviewed changes

sjy982 and others added 4 commits November 3, 2025 14:53

Update core/src/main/java/com/linecorp/armeria/client/websocket/WebSo…

d5036c9

…cketClientBuilder.java Co-authored-by: Ikhun Um <[email protected]>

Update core/src/main/java/com/linecorp/armeria/client/websocket/WebSo…

719047c

…cketSession.java Co-authored-by: Ikhun Um <[email protected]>

Update core/src/main/java/com/linecorp/armeria/internal/server/websoc…

cc79360

…ket/DefaultWebSocketService.java Co-authored-by: Ikhun Um <[email protected]>

Merge branch 'main' into feat/ws-timeout-support

01916d4

jrhee17 approved these changes Nov 4, 2025

View reviewed changes

sjy982 and others added 3 commits November 7, 2025 17:16

Update core/src/main/java/com/linecorp/armeria/common/InboundComplete…

e7b4c0e

…Exception.java Co-authored-by: jrhee17 <[email protected]>

Update core/src/main/java/com/linecorp/armeria/common/websocket/WebSo…

06845ae

…cketIdleTimeoutException.java Co-authored-by: jrhee17 <[email protected]>

fix: Apply review

7315e52

sjy982 requested a review from jrhee17 November 7, 2025 08:41

jrhee17 modified the milestones: 1.34.0, 1.35.0 Nov 24, 2025

Add streamTimeout option to WebSocketService/WebSocketClient; send close frame on inbound failure; clean up related resources #6357

Are you sure you want to change the base?

Add streamTimeout option to WebSocketService/WebSocketClient; send close frame on inbound failure; clean up related resources #6357

Uh oh!

Conversation

sjy982 commented Aug 17, 2025

Uh oh!

github-actions bot commented Aug 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Build Scan® (commit: 7315e52)

Uh oh!

jrhee17 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ikhoon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jrhee17 Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

sjy982 Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

jrhee17 Nov 12, 2025

Choose a reason for hiding this comment

Uh oh!

sjy982 Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

jrhee17 Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ikhoon Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

github-actions bot commented Aug 17, 2025 •

edited

Loading

🔍 Build Scan® (commit: `7315e52`)

codecov bot commented Aug 28, 2025 •

edited

Loading

jrhee17 Nov 18, 2025 •

edited

Loading