Allow MCPRemoteProxy to work without upstream or client auth

[aron-muon]

Summary

MCPRemoteProxy always injected upstream IdP tokens when the embedded auth server was configured, and the transparent proxy always emitted X-Forwarded-Host to remote upstreams. Two consequences:

1. Public/anonymous upstreams couldn't be proxied with client OAuth. Operators who want to authenticate clients (Okta, GitHub, etc.) but proxy a public MCP server (e.g. documentation APIs) had no way to skip the upstream token injection.
2. Cross-host redirect loops. Upstreams that build redirect URLs from X-Forwarded-Host would echo the proxy's hostname back, creating loops the transparent proxy then refused to follow ("refusing cross-host redirect from remote MCP server").

This PR addresses both with two focused changes per @ChrisJBurns's review (#4168 (review)).

What changed (2 logical changes):

1. disableUpstreamTokenInjection flag on EmbeddedAuthServerConfig. When true, the embedded auth server still runs the OAuth flow for clients but skips upstream token swapping. A strip-auth middleware removes the client's ToolHive JWT from the Authorization header before forwarding so the bearer token doesn't leak to the upstream either.

2. Skip SetXForwarded() for remote upstreams in the transparent proxy. Prevents X-Forwarded-Host from leaking the proxy hostname to third-party servers. Adds debug logging on outbound requests and upstream responses.

Dropped from earlier revisions of this PR:

• Optional oidcConfig — @ChrisJBurns confirmed this is already resolved on main: both OIDCConfig and OIDCConfigRef are pointer types and resolveAndAddOIDCConfig returns (nil, nil) cleanly when both are absent. The OIDC resolver, AddOIDCConfigOptions, and controller validation all handle nil without error.
• Custom forwardFollowingRedirects — main now has a more secure redirect-following implementation with cross-host and HTTPS-downgrade guards (addresses Claude Bot's earlier security finding on this PR about the Authorization header being cloned across hosts).

Rebased onto current main — clean cherry-pick after dropping Goal 2; conflicts resolved against the v1alpha1→v1beta1 rename.

Type of change

• New feature

Test plan

• Unit tests (go test ./cmd/thv-operator/... ./pkg/runner/... ./pkg/transport/proxy/transparent/... ./pkg/authserver/...) — all pass
• Build verification (go build ./...) — clean
• go vet ./cmd/thv-operator/... — clean
• CRDs regenerated (controller-gen v0.17.3) and committed

API Compatibility

• This PR does not break the v1beta1 API. The added field (disableUpstreamTokenInjection) is optional with a false default, matching the existing behavior.

Changes

File	Change
cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go	Add DisableUpstreamTokenInjection to EmbeddedAuthServerConfig
cmd/thv-operator/pkg/controllerutil/authserver.go	Wire CRD field through to runtime RunConfig
cmd/thv-operator/pkg/controllerutil/authserver_test.go	Tests for wiring + default value
pkg/authserver/config.go	Add DisableUpstreamTokenInjection to runtime RunConfig
pkg/runner/middleware.go	Add strip-auth middleware; skip upstream swap when injection disabled
pkg/runner/middleware_test.go	Tests for strip-auth and upstream-swap selection
pkg/transport/proxy/transparent/transparent_proxy.go	Skip SetXForwarded() for remote upstreams; add debug logging
deploy/charts/operator-crds/{files,templates}/...	Regenerated CRD YAMLs (controller-gen v0.17.3)
docs/operator/crd-api.md, docs/server/{docs.go,swagger.{json,yaml}}	Regenerated docs

Does this introduce a user-facing change?

Yes. One new capability for MCPExternalAuthConfig (type embeddedAuthServer):

• spec.embeddedAuthServer.disableUpstreamTokenInjection: true — authenticates MCP clients via the embedded OAuth server but does not inject upstream IdP tokens or forward the client's ToolHive JWT to the backend. Useful for proxying public MCP servers (documentation, search, etc.) where you want client auth but no upstream auth.

The X-Forwarded-Host change is a bug fix; no API surface change.

🤖 Generated with Claude Code

[github-actions]

Large PR Detected

This PR exceeds 1000 lines of changes and requires justification before it can be reviewed.

How to unblock this PR:

Add a section to your PR description with the following format:

## Large PR Justification

[Explain why this PR must be large, such as:]
- Generated code that cannot be split
- Large refactoring that must be atomic
- Multiple related changes that would break if separated
- Migration or data transformation

Alternative:

Consider splitting this PR into smaller, focused changes (< 1000 lines each) for easier review and reduced risk.

See our Contributing Guidelines for more details.

---

This review will be automatically dismissed once you add the justification section.

[codecov]

Codecov Report

❌ Patch coverage is 62.85714% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.91%. Comparing base (9211a36) to head (49533c1).

Files with missing lines	Patch %	Lines
pkg/runner/middleware.go	31.57%	12 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4168      +/-   ##
==========================================
+ Coverage   67.86%   67.91%   +0.05%     
==========================================
  Files         610      610              
  Lines       62522    62556      +34     
==========================================
+ Hits        42431    42488      +57     
+ Misses      16910    16884      -26     
- Partials     3181     3184       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Large PR justification has been provided. Thank you!

[github-actions]

✅ Large PR justification has been provided. The size review has been dismissed and this PR can now proceed with normal review.

[ChrisJBurns]

@aron-muon hey bud, just got back from leave, for reference this RFC has been raised that aims to provide a new CRD that will eventually lead to the deprecation and removal of MCPRemoteProxy

[aron-muon]

@aron-muon hey bud, just got back from leave, for reference this RFC has been raised that aims to provide a new CRD that will eventually lead to the deprecation and removal of MCPRemoteProxy

Very exciting!

[lujunsan]

@claude review the changes and check if this overlaps with the MCPOIDCConfig CRD changes that have been added recently.

[claude]

Claude Code is working…

I'll analyze this and get back to you.

View job run

[claude]

Regarding the MCPOIDCConfig overlap question: I don't find a MCPOIDCConfig CRD type in the current main branch, so I can't verify overlap — this likely refers to a pending or in-progress change that should be compared manually. A security issue was also identified in the new redirect-following logic (see inline comment) that should be addressed before merge.

Extended reasoning...

Overview

This PR adds three features to MCPRemoteProxy: optional OIDC (anonymous access), a DisableUpstreamTokenInjection flag with strip-auth middleware, and a new forwardFollowingRedirects function in the transparent proxy. It touches 27 files including CRD types, controller logic, middleware, and the core transparent proxy transport.

Security risks

The bug hunting system identified a real security regression in forwardFollowingRedirects: the Authorization header (containing the client JWT or upstream IdP access token) is cloned unconditionally into redirect requests, including cross-host redirects. Go's standard http.Client has protected against this since Go 1.12 by stripping Authorization/Cookie headers on cross-host redirects — this implementation omits that check. This is a credential exfiltration risk when the upstream issues 3xx redirects to attacker-controlled or unintended external hosts.

Level of scrutiny

High. The PR modifies security-sensitive proxy transport code, introduces new auth middleware, and changes a required CRD field to optional (breaking schema change). The transparent proxy redirect logic in particular is new and complex with insufficient test coverage (41% patch coverage reported by Codecov).

Other factors

A human reviewer (ChrisJBurns) noted in March that an RFC is being drafted to deprecate/replace MCPRemoteProxy. A reviewer (lujunsan) also specifically requested a check against MCPOIDCConfig CRD changes; no MCPOIDCConfig type exists in main, so that overlap cannot be verified automatically. The combination of an unresolved design direction, a security bug in new proxy code, and the explicit overlap question warrants human review before merge.

[aron-muon]

@claude review the changes and check if this overlaps with the MCPOIDCConfig CRD changes that have been added recently.

Cleanly rebased now! @lujunsan

[ChrisJBurns]

Hey @aron-muon, thanks for keeping this rebased! I did a deep dive into the current state of main to see which parts of this PR are still needed.

Goal 2 (optional oidcConfig / anonymous proxy) is already resolved on main. Both OIDCConfig (which is being removed soon in favour of the dedicated CRD) and OIDCConfigRef are optional pointers, and resolveAndAddOIDCConfig() gracefully returns (nil, nil) when both are absent. The OIDC resolver, AddOIDCConfigOptions, and controller validation all handle the nil case without error. So that piece can be dropped.

Goals 1 and 3 are still unresolved:

• DisableUpstreamTokenInjection — no equivalent exists. There's currently no way to run the embedded auth server for client-facing OAuth without also injecting tokens upstream.
• SetXForwarded() skipping for remote upstreams — still called unconditionally in transparent_proxy.go. The codebase has workarounds for specific symptoms (AWS SigV4, SSE URL rewriting), but the root cause remains.

It's also worth noting that for users who don't need the proxy pod at all — they just want vMCP to call a remote server with optional auth — MCPServerEntry + VirtualMCPServer already avoids all of these issues since there's no intermediate proxy setting X-Forwarded headers or injecting tokens. Goals 1 and 3 are specifically relevant when you need the proxy pod (e.g., for client-facing embedded auth server OAuth flows).

Would you be open to splitting this into a focused PR covering just goals 1 and 3? That would make it smaller and easier to review. Happy to help coordinate.

[reyortiz3]

hi @aron-muon! 👋 just checking on this PR if there is any other help needed. Thanks!

[aron-muon]

Hey @aron-muon, thanks for keeping this rebased! I did a deep dive into the current state of main to see which parts of this PR are still needed.

Goal 2 (optional oidcConfig / anonymous proxy) is already resolved on main. Both OIDCConfig (which is being removed soon in favour of the dedicated CRD) and OIDCConfigRef are optional pointers, and resolveAndAddOIDCConfig() gracefully returns (nil, nil) when both are absent. The OIDC resolver, AddOIDCConfigOptions, and controller validation all handle the nil case without error. So that piece can be dropped.

Goals 1 and 3 are still unresolved:

• DisableUpstreamTokenInjection — no equivalent exists. There's currently no way to run the embedded auth server for client-facing OAuth without also injecting tokens upstream.
• SetXForwarded() skipping for remote upstreams — still called unconditionally in transparent_proxy.go. The codebase has workarounds for specific symptoms (AWS SigV4, SSE URL rewriting), but the root cause remains.

It's also worth noting that for users who don't need the proxy pod at all — they just want vMCP to call a remote server with optional auth — MCPServerEntry + VirtualMCPServer already avoids all of these issues since there's no intermediate proxy setting X-Forwarded headers or injecting tokens. Goals 1 and 3 are specifically relevant when you need the proxy pod (e.g., for client-facing embedded auth server OAuth flows).

Would you be open to splitting this into a focused PR covering just goals 1 and 3? That would make it smaller and easier to review. Happy to help coordinate.

Updated with just goal 1 and 3!

[aron-muon]

hi @aron-muon! 👋 just checking on this PR if there is any other help needed. Thanks!

Ready for re-review, sorry for the delay!