Support an authenticating egress proxy for outbound HTTP clients

[ewels]

Summary

Addresses nextflow-io/nextflow#7305.

An on-prem Wave deployment behind a corporate egress proxy could not reach container registries or Seqera Platform: the outbound java.net.http.HttpClient instances created by HttpClientFactory had no proxy selector and — crucially — no proxy authenticator (java.net.http.HttpClient ignores Authenticator.setDefault(); the authenticator must be set on the client builder).

This PR adds egress proxy support to HttpClientFactory, the single chokepoint through which all outbound clients are created (ProxyClient, StreamServiceImpl, registry auth/lookup, Tower connector, etc.). It mirrors the approach used to fix the same problem in Nextflow (nextflow-io/nextflow#7331): the proxy selector and proxy-scoped authenticator come from the shared io.seqera:lib-httpx library (HxProxyConfig), so both products share the same proxy semantics.

Changes

• io.seqera:lib-httpx:2.6.0 (new dependency): provides HxProxyConfig — per-protocol HTTP/HTTPS proxy entries with a ProxySelector honouring the no-proxy list, and an Authenticator scoped to RequestorType.PROXY and the configured proxy host/port so credentials are only ever released to the proxy itself.
• ProxyConfig (new, modelled on Nextflow's nextflow.util.ProxyConfig): parses [http[s]://][user:pass@]host[:port] proxy URIs (percent-encoded credentials supported, with + shielded from the form-decoding rule) and resolves the HxProxyConfig from either an explicit URI or the HTTPS_PROXY/HTTP_PROXY/NO_PROXY environment variables (per-protocol proxies supported via env vars).
• HttpClientConfig (now a @Context bean): owns the new settings wave.httpclient.proxy.uri, .username, .password, .no-proxy alongside the existing wave.httpclient.* options, resolves them at startup — falling back to the environment variables when not set — and applies the result to HttpClientFactory.
• HttpClientFactory: clients are built with HxProxyConfig.toProxySelector()/.toAuthenticator() when a proxy is configured, via a setProxyConfig(...) bridge that invalidates the cached clients. With no proxy configured, no .proxy(...) call is made and behaviour is unchanged (requests still route via ProxySelector.getDefault(), i.e. -Dhttps.proxyHost et al. keep working).
• Docs: new Egress proxy section in docs/install/reference.md, including an explicit statement of scope (the Wave process's internal HTTP clients; Micronaut declarative clients, AWS SDK clients, and out-of-process build/scan/blob-cache jobs have their own proxy configuration paths).

No-proxy semantics follow lib-httpx (same as Nextflow): bare host names match the host and its sub-domains, .domain/*.domain suffixes match sub-domains only, * disables proxying, and loopback targets always bypass the proxy. CIDR entries are not supported.

The disabledSchemes dependency

By default the JDK ships jdk.http.auth.tunneling.disabledSchemes=Basic, which silently blocks Basic authentication on the HTTPS CONNECT tunnel — i.e. exactly the path used to reach https:// registries through a proxy. When a proxy with credentials is configured, HttpClientConfig defaults both jdk.http.auth.tunneling.disabledSchemes and jdk.http.auth.proxying.disabledSchemes to empty at startup, unless the operator has already set them, in which case the operator's value wins.

Because these properties are read once by the JDK HTTP stack, setting them explicitly on the JVM is the belt-and-braces alternative for containerised deploys, documented in docs/install/reference.md:

JAVA_TOOL_OPTIONS="-Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes="

Tests

• ProxyConfigTest: URI/credential parsing, env-var resolution (per-protocol proxies, lowercase variants, credentials), no-proxy matching via the selector (suffix/wildcard/loopback), authenticator scoping (requestor type, host, port).
• HttpClientFactoryProxyTest: integration test against an in-process authenticating proxy (FakeProxyServer, supporting Basic auth, plain forwarding and CONNECT tunnelling), with target servers bound to a non-loopback address since loopback targets always bypass the proxy:
  • 407 surfaced when the proxy requires auth and no credentials are configured — both redirect policies;
  • successful authenticated requests through the proxy — both redirect policies;
  • no-proxy list bypass;
  • HTTPS CONNECT tunnelling with Basic proxy auth against a local TLS server (self-signed keystore generated on the fly with keytool), which requires -Djdk.http.auth.tunneling.disabledSchemes= — added to the test task in build.gradle.

All 44 new tests pass. Pre-existing network-dependent suites fail identically with and without this change in the sandbox environment (registry credentials/egress policy), i.e. no regression.

Backwards compatibility

No proxy configured (neither settings nor environment variables) → no behaviour change. Note that deployments that already export HTTPS_PROXY/HTTP_PROXY will now have those honoured by Wave's outbound clients; NO_PROXY is honoured accordingly.

🤖 Generated with Claude Code

https://claude.ai/code/session_018zJyj9FkzUpMNuH51ScqAQ

[ewels]

Note that another approach here could be to pull in libseqera to do more of the work and avoid having a duplicate fix to the other linked PRs. As that's a bigger change with wider architectural decisions to make I will leave that alone for now though.

[ewels]

Following up on the comment above: this PR has now been reworked to do exactly that — it pulls in the shared libseqera implementation instead of carrying a duplicate fix.

• The branch was rebased on current master (resolving the conflicts from the docs restructure and the kebab-case config key rename).
• The bespoke ProxySelector/Authenticator/no-proxy matching has been deleted and replaced with io.seqera:lib-httpx:2.6.0 (HxProxyConfig) — the same library and version-line used to fix this in Nextflow via Support authenticated forward HTTP/HTTPS proxies across the client stack nextflow-io/nextflow#7331 — so Wave and Nextflow now share identical proxy semantics.
• What remains Wave-side is only the resolution layer: the wave.httpclient.proxy.* settings with HTTPS_PROXY/HTTP_PROXY/NO_PROXY env-var fallback (ProxyConfig, modelled on Nextflow's nextflow.util.ProxyConfig), the HttpClientFactory wiring, and the disabledSchemes startup defaulting.

One behavioural note from adopting the shared library: no-proxy entries follow lib-httpx semantics (bare hostnames also match sub-domains, .domain/*.domain match sub-domains only, loopback always bypasses; CIDR entries are not supported). The PR description has been updated to reflect all of this.

---

Generated by Claude Code