fix(payments): trust Host header for checkout return URLs, not req.nextUrl.origin (#479)

[guillermoscript]

What & why

Found during the live-credentialed portion of #479 (PayPal sandbox QA follow-up to #466 / PR #478). Independent of the categoryMap fix in #490 — this is a checkout-redirect bug, not a settings-page bug, and it affects every non-Stripe provider that round-trips through a hosted checkout, not just PayPal.

The bug

app/api/payments/checkout/route.ts and app/api/payments/paypal/capture/route.ts both built the buyer's return origin like this:

process.env.NODE_ENV === 'development'
  ? req.nextUrl.origin
  : forwardedHost ? `https://${forwardedHost}` : req.nextUrl.origin

req.nextUrl.origin does not reflect the incoming Host header in development — it resolves to the Next.js dev server's own bind address (localhost:PORT), regardless of which tenant subdomain the request actually came in on.

Live reproduction on the code-academy tenant: a PayPal one-time checkout built its return_url from req.nextUrl.origin, so after the buyer approved the payment on PayPal's sandbox, they were redirected to localhost:3000/checkout/success?transactionId=… instead of code-academy.lvh.me:3000/checkout/success?…. Since their session cookie is scoped to the tenant subdomain, localhost had no session, and the buyer was bounced to /auth/login — even though the payment itself succeeded (confirmed via the transactions row flipping to successful and the PAYMENT.CAPTURE.COMPLETED webhook processing correctly in webhook_events). The purchase was never actually lost, but the buyer's browser landed on a broken, illegitimate-looking page after paying.

The same pattern already exists correctly in app/api/stripe/connect/route.ts, which derives the origin from x-forwarded-host / host headers directly rather than trusting req.nextUrl.origin in development.

The fix

Both routes now resolve the origin the same way stripe/connect/route.ts does:

const host = req.headers.get('x-forwarded-host') ?? req.headers.get('host') ?? req.nextUrl.host
const proto = req.headers.get('x-forwarded-proto') ?? req.nextUrl.protocol.replace(':', '')
const appUrl = `${proto}://${host}`

No environment-conditional branching — this resolves correctly in both dev and production, since the Host header always carries the actual authority the browser dialed.

Test plan

• npx tsc --noEmit → clean.
• npx eslint on both changed files → 0 errors.
• npx vitest run → full suite, 218/218 pass (no existing tests target these specific route handlers; nothing regressed).
• Live-verified against the real PayPal sandbox on the code-academy tenant subdomain, both before and after the fix:
  • Before fix: reproduced the bug exactly — buyer redirected to localhost:3000/checkout/success after PayPal approval, bounced to /auth/login on a different origin than their session.
  • After fix: a fresh PayPal one-time purchase completed and redirected the buyer back to code-academy.lvh.me:3000/dashboard/student/courses/… correctly, with the enrollment created and the PAYMENT.CAPTURE.COMPLETED webhook processed.
  • After fix: a fresh PayPal subscription purchase completed and redirected back to the correct tenant subdomain, with the subscription row created (subscription_status: active, correct provider_subscription_id) and both BILLING.SUBSCRIPTION.ACTIVATED and the immediate renewal webhook processed.
  • Screenshot attached below.

Screenshot

PayPal one-time checkout completing and landing back on the tenant subdomain (code-academy.lvh.me:3000) after the fix — attached as a PR comment.

[guillermoscript]

Visual verification

A fresh PayPal sandbox one-time purchase after the fix, showing the buyer correctly landed back on the tenant subdomain (code-academy.lvh.me:3000) after approving the payment on PayPal — enrolled in the course, tenant-branded sidebar, no bounce to /auth/login:

Before the fix, this same flow redirected the buyer to localhost:3000/checkout/success instead — a different origin than their tenant-subdomain session — bouncing them to /auth/login even though the payment had succeeded.