feat: send HTTP/2 PING keepalives on the Bedrock provider - #5213
feat: send HTTP/2 PING keepalives on the Bedrock provider#5213jeremym-tanium wants to merge 4 commits into
Conversation
📝 WalkthroughSummary by CodeRabbit
WalkthroughAdds an optional HTTP/2 PING interval to Bedrock network configuration. The interval is serialized, schema-validated, tested, and applied to the Bedrock transport only when HTTP/2 is enforced and the interval is positive. Frontend schemas and types now represent the setting. Materialized-view interval validation also supports explicit disabling. ChangesBedrock HTTP/2 keepalive
Materialized-view interval schema
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related issues
Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant NetworkConfig
participant NewBedrockProvider
participant HTTP2Transport
participant Intermediary
NetworkConfig->>NewBedrockProvider: HTTP2PingIntervalInSeconds
NewBedrockProvider->>HTTP2Transport: configure SendPingTimeout
HTTP2Transport->>Intermediary: send HTTP/2 PING
Intermediary-->>HTTP2Transport: acknowledge HTTP/2 PING
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Confidence Score: 5/5This looks safe to merge.
Important Files Changed
Reviews (8): Last reviewed commit: "fix: make Bedrock HTTP/2 keepalive ping ..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
504e3f9 to
6356afb
Compare
|
Addressed the schema finding: added |
6356afb to
9e85f2e
Compare
|
Question for Bifrost devs: Initially when I created the PR, I enabled the HTTP/2 PING keepalives by default with a reasonable interval, but I backed off that to preserve the current behavior but allowed configurability. The downside of that though is that other users could potentially experience these expected EOF stream timeout errors, debug, and then discover that there's a config knob. Would you prefer enabling these keepalives by default? |
5491cac to
52e2db0
Compare
|
Rebased onto the latest |
|
Rebased onto the latest |
52e2db0 to
d97f10a
Compare
|
Rebased onto the latest |
d97f10a to
52eede6
Compare
The merge-base changed after approval.
|
|
52eede6 to
7c4a867
Compare
7c4a867 to
a7ac3d0
Compare
The merge-base changed after approval.
When enforce_http2 is set, configure http.HTTP2Config.SendPingTimeout so the client sends PING frames while a streaming or unary response is idle, keeping a long-lived connection from being closed by an intermediary idle timeout (which the AWS EventStream decoder would otherwise surface as an "unexpected EOF"). The interval is configurable via network_config http2_keepalive_interval_in_seconds (default 30s).
Mirror http2_ping_interval_in_seconds into the chart values.schema.json networkConfig def so Helm deployments can set it.
There was no HTTP/2 ping keepalive before this change, so the 30s default-fill was an arbitrary imposition. Treat 0 as disabled (matching net/http's own SendPingTimeout semantics) and only send pings when a positive interval is configured, keeping enforce_http2 orthogonal to keepalive.
a7ac3d0 to
bf89299
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/providers/bedrock/bedrock.go`:
- Around line 128-131: Update the UI NetworkConfig type in config.ts to declare
the http2_ping_interval_in_seconds property used by the Bedrock transport
configuration, with the appropriate numeric/optional contract so UI
serialization preserves the HTTP/2 keepalive value.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 514a3de3-06cf-4bf4-aba7-663b70272b76
📒 Files selected for processing (6)
core/changelog.mdcore/providers/bedrock/bedrock.gocore/schemas/provider.gocore/schemas/serialization_test.gohelm-charts/bifrost/values.schema.jsontransports/config.schema.json
🚧 Files skipped from review as they are similar to previous changes (2)
- helm-charts/bifrost/values.schema.json
- core/schemas/serialization_test.go
The TS NetworkConfig interface and its three Zod mirrors (providerForm's NetworkConfigSchema, and schemas.ts's networkConfigSchema / networkFormConfigSchema) were missing the field added alongside enforce_http2 in the Go schema, so a value round-tripped through those validators would be silently stripped.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ui/lib/schemas/providerForm.ts`:
- Line 46: Update the http2_ping_interval_in_seconds field in the provider form
schema to include the shared 3600-second maximum, preserving its integer,
non-negative, and optional constraints and using the same validation message as
the shared network schema.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: a757472f-0479-4772-aaad-004e0ac04bef
📒 Files selected for processing (3)
ui/lib/schemas/providerForm.tsui/lib/types/config.tsui/lib/types/schemas.ts
| keep_alive_timeout_in_seconds: z.number().int().min(1).max(3600).optional(), | ||
| max_conns_per_host: z.number().int().min(1).max(10000).optional(), | ||
| enforce_http2: z.boolean().optional(), | ||
| http2_ping_interval_in_seconds: z.number().int().min(0).optional(), |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Keep the provider-form bound consistent with the shared network schema.
This accepts any non-negative integer, while ui/lib/types/schemas.ts caps the same setting at 3600 seconds. Values above 3600 can therefore pass this schema but fail shared validation later. Add the same .max(3600, ...) constraint here.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ui/lib/schemas/providerForm.ts` at line 46, Update the
http2_ping_interval_in_seconds field in the provider form schema to include the
shared 3600-second maximum, preserving its integer, non-negative, and optional
constraints and using the same validation message as the shared network schema.
4 participants
Summary
When a Bedrock stream goes silent for tens of seconds (model "thinking"), an intermediary (reverse proxy / load balancer / service mesh) with an idle timeout shorter than the gap can sever the connection, surfacing as
io.ErrUnexpectedEOF. This adds client-initiated HTTP/2 PING keepalives on the Bedrock provider transport so a quiet stream stays warm.Changes
core/providers/bedrock/bedrock.go: whennetwork_config.enforce_http2is set and a positivehttp2_ping_interval_in_secondsis configured, set the transport'shttp.HTTP2Config{SendPingTimeout}(Go 1.24+ stdlib) to send a PING after that idle interval. Off by default (no ping) to keepenforce_http2orthogonal to keepalive.core/schemas/provider.go: addhttp2_ping_interval_in_secondstoNetworkConfig(opt-in;0= disabled, matching net/http's ownSendPingTimeoutsemantics; mirrored in the Marshal/Unmarshal aliases).core/schemas/serialization_test.go: unit test for the field round-trip + opt-in (0 = off) behavior.Type of change
How to test
cd core && go test ./schemas/ -run TestNetworkConfig_HTTP2PingInterval -count=1. Also validated end-to-end against real Bedrock behind an intermediary with a short idle timeout: HTTP/2 negotiated, PING frames emitted at the interval, stream survives a long thinking gap.Affected areas
core / providers / bedrock; core / schemas
Breaking changes
None — gated on the existing
enforce_http2flag; default off.Related issues
Closes #5211
Security considerations
None.