fix(api): dont 500 on cross dc response

[MasterPtato]

Fixes RVT-6105

Description

Please include a summary of the changes and the related issue. Please also include relevant motivation and context.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[railway-app]

🚅 Deployed to the rivet-pr-4449 environment in rivet-frontend

Service	Status	Web	Updated (UTC)
frontend-cloud	😴 Sleeping (View Logs)	Web	Mar 24, 2026 at 8:48 am
frontend-inspector	😴 Sleeping (View Logs)	Web	Mar 19, 2026 at 12:12 pm
website	❌ Build Failed (View Logs)	Web	Mar 17, 2026 at 10:41 pm
mcp-hub	✅ Success (View Logs)	Web	Mar 17, 2026 at 10:40 pm
ladle	❌ Build Failed (View Logs)	Web	Mar 17, 2026 at 10:39 pm

[MasterPtato]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat(pb): actors v3 #4463
• feat(ups): implement queue subs #4486
• feat(cache): add in flight deduping #4459
• fix(cache): clean up lib #4457
• fix(config): allow configuring topo dcs via map, fix pg ssl mode config #4456
• fix(api): dont 500 on cross dc response #4449 👈 (View in Graphite)
• chore: update byte units on frontend #4444
• fix: runner alloc idx logic, api auth for actor get #4443
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[linear]

RVT-6105 Cross-DC proxy escalates 400 client errors to 500 internal errors

[pkg-pr-new]

More templates

• actor-actions
• actor-actions-vercel
• ai-agent
• ai-agent-vercel
• example-ai-and-user-generated-actors-freestyle
• chat-room
• chat-room-vercel
• example-cloudflare-workers
• example-cloudflare-workers-hono
• collaborative-document
• collaborative-document-vercel
• cross-actor-actions
• cross-actor-actions-vercel
• example-cursors
• example-cursors-raw-websocket
• example-cursors-raw-websocket-vercel
• example-cursors-vercel
• custom-serverless
• example-elysia
• example-experimental-durable-streams-ai-agent
• example-experimental-durable-streams-ai-agent-vercel
• geo-distributed-database
• geo-distributed-database-vercel
• hello-world
• hello-world-vercel
• example-hono
• hono-react
• hono-react-vercel
• example-kitchen-sink
• example-kitchen-sink-vercel
• multi-region-vercel
• multiplayer-game
• example-multiplayer-game-patterns
• example-multiplayer-game-patterns-vercel
• multiplayer-game-vercel
• native-websockets
• native-websockets-vercel
• next-js
• per-tenant-database
• per-tenant-database-vercel
• example-raw-fetch-handler
• example-raw-fetch-handler-vercel
• example-raw-websocket-handler
• example-raw-websocket-handler-proxy
• example-raw-websocket-handler-proxy-vercel
• example-raw-websocket-handler-vercel
• example-react
• example-react-vercel
• sandbox
• sandbox-coding-agent
• sandbox-coding-agent-vercel
• sandbox-vercel
• scheduling
• scheduling-vercel
• example-sqlite-drizzle
• example-sqlite-raw
• state
• state-vercel
• example-stream
• example-stream-vercel
• example-trpc

@rivetkit/cloudflare-workers

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/cloudflare-workers@4449

@rivetkit/framework-base

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/framework-base@4449

@rivetkit/next-js

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/next-js@4449

@rivetkit/react

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/react@4449

rivetkit

pnpm add https://pkg.pr.new/rivet-dev/rivet/rivetkit@4449

@rivetkit/sql-loader

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/sql-loader@4449

@rivetkit/sqlite-vfs

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/sqlite-vfs@4449

@rivetkit/traces

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/traces@4449

@rivetkit/workflow-engine

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/workflow-engine@4449

@rivetkit/virtual-websocket

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/virtual-websocket@4449

@rivetkit/engine-runner

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/engine-runner@4449

@rivetkit/engine-runner-protocol

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/engine-runner-protocol@4449

commit: 876713f

[claude]

PR Review: fix(api): dont 500 on cross dc response

Summary

This is a minimal, well-targeted fix in engine/packages/api-util/src/lib.rs in reqwest_to_axum_response.

Before: When a remote DC returned a non-2xx response, anyhow::bail! propagated an Err up the call stack, causing the API handler to respond with HTTP 500.

After: Logs the error via tracing::error! and falls through to return the actual remote response, preserving the original status code and body.

---

Feedback

Correctness: ✅

The fix is correct. reqwest_to_axum_response should convert a reqwest response to an axum response — a non-2xx upstream status is not a programming error, it's a valid response to be forwarded. Bailing here was the wrong abstraction.

Logging: ✅

The structured logging conversion is well done and aligns with CLAUDE.md: "Do not format parameters into the main message, instead use tracing's structured logging."

Minor observation:

?status (Debug) prints as StatusCode(404) while %status (Display) would print as 404 Not Found, which is slightly more readable. Not blocking, just worth noting.

---

Verdict

✅ Looks good to merge. Simple, correct fix with minimal blast radius.

[claude]

PR Review: fix(api) dont 500 on cross dc response

This is a small, focused bug fix that correctly addresses the issue of cross-DC responses being converted to 500 errors.

Change Summary: In reqwest_to_axum_response, when a remote DC returned a non-2xx response, anyhow::bail!() was called, which propagated as an internal server error (500) instead of forwarding the actual response status and body to the client. The fix logs the error and allows the response to pass through with its original status code.

Whats Good: reqwest_to_axum_response is intended as a raw pass-through so forwarding the original status code and body is the right approach. The new tracing::error! call correctly follows the project logging conventions. The body_bytes borrow via String::from_utf8_lossy is correctly scoped within the if block.

Minor Suggestions: (1) Consider using %status (Display) instead of ?status (Debug) for the status code - Display outputs 404 Not Found while Debug adds extra formatting noise. (2) tracing::error! is reasonable but tracing::warn! might reduce log noise if 404s are expected in normal operation. (3) The checklist items in the PR description are unchecked - worth filling in for completeness.

Overall this is a correct and minimal fix. LGTM.

[claude]

PR Review: fix(api): dont 500 on cross dc response

Overall: Clean, correct, minimal fix.

What the change does

In reqwest_to_axum_response, when a remote DC returns a non-2xx response, the old code called anyhow::bail! which propagated as a 500, discarding the actual remote response. The fix replaces that with a tracing::error! log and lets the function continue to forward the remote DC's actual status code, headers, and body to the caller.

Correctness

The fix is correct. The function's job is to convert a reqwest::Response into an Axum Response — including error responses. Bailing on non-2xx statuses was the bug: it prevented legitimate error responses (404, 429, etc.) from being forwarded to the client.

Logging style

The new tracing::error! call matches project conventions correctly:

tracing::error!(?status, ?ray_id, %body_text, "remote dc returned error");

Structured fields with ? for Debug, % for Display, lowercase message — all good.

Minor concern: log level

error is used unconditionally for all non-2xx responses, including 4xx client errors like 404 or 400. This may produce noisy error logs for expected client errors. Consider differentiating by status class:

if status.is_server_error() {
    tracing::error!(?status, ?ray_id, %body_text, "remote dc returned error");
} else {
    tracing::warn!(?status, ?ray_id, %body_text, "remote dc returned error");
}

Or use warn consistently since the caller receives the full response and can decide how to handle it. This is a minor nit — the core fix is sound either way.