fix(security): pin integration api_call to the SSRF-validated IP#5514
fix(security): pin integration api_call to the SSRF-validated IP#5514ashvinctrl wants to merge 2 commits into
Conversation
execute_api_call validated the outbound URL with check_outbound_url (odysseus-dev#5145) but then opened a plain httpx client that re-resolves the host at connect time. A base_url host that answers with a public IP for the guard and rebinds to 169.254.169.254 for the connect slips past the check and reaches cloud metadata with the integration's auth headers attached — the same DNS-rebinding TOCTOU already closed for web fetch (odysseus-dev#704) and webhook delivery (odysseus-dev#5147). Resolve the host once through the guard's own resolver, remember the validated IPs, and pin the request connection to one of them via a local async transport (SNI and Host stay the original hostname). The per-module transport mirrors the ones web fetch and webhook delivery already carry.
This comment was marked as off-topic.
This comment was marked as off-topic.
RaresKeY
left a comment
There was a problem hiding this comment.
I checked the latest head and found two transport regressions that should be addressed before merge. The SSRF pin itself held in real socket probes, but the new transport drops normal address fallback and changes the HTTPS trust configuration.
Findings
issue (networking): Preserve fallback across the validated DNS addresses
-
Problem: The guard validates every DNS answer, but
_first_resolved_ip()retains only the first and the pinned backend can connect only to that address. With an approved but unreachable first address and a reachable second address, the full integration call fails instead of trying the remaining approved route. -
Impact: Valid multi-address and dual-stack integrations can lose all API calls and connection tests solely because of resolver ordering, a stale record, or an unavailable address family, even though another address from the same SSRF-approved set is healthy.
-
Ask: Carry the complete validated address set into the backend and perform connect-level fallback or racing only among those approved literals, without re-resolving the hostname or replaying the application request. Keep fallback inside the existing connect-time budget and add a real first-dead/second-live socket regression that preserves the original Host/TLS SNI.
-
Location:
src/integrations.py:550
issue (TLS): Preserve HTTPX's CA trust configuration in the pinned transport
-
Problem:
_PinnedAsyncTransportbuilds the pool withssl.create_default_context(), while the previous default HTTPX client uses HTTPX's CA context. Without an environment CA override, this switches HTTPS integrations from HTTPX's default CA set to the host system roots. -
Impact: Certificate chains trusted only by the prior HTTPX context can stop verifying, taking all calls and connection tests for an otherwise valid HTTPS integration offline; roots present only in the system bundle also become newly trusted.
-
Ask: Build or inject the same SSL context the default HTTPX client would have used, including supported environment CA overrides, while keeping hostname verification and SNI bound to the original URL host. Add focused coverage for the intended trust source.
-
Location:
src/integrations.py:416
Validation
- Ran: focused integration, URL-safety, and existing socket-pin tests; real pinned-address, Host/path/query/auth, redirect, gzip, first-dead/second-live, and CA-context comparison checks.
- Not run: full local suite or live external dual-stack/TLS, custom-CA, and corporate-proxy testing.
…nned transport RaresKeY review on odysseus-dev#5514: - P2 (networking): the pin kept only the first resolved IP, so an approved but unreachable first address failed the whole call even when another validated address was healthy. check_outbound_url only returns ok when *every* resolved IP is safe, so the full set is guard-approved — carry all of them and let the pinned backend fall back address-by-address at connect time. Fallback shares one connect deadline (no N*timeout blow-up), never re-resolves, and never replays the request. Adds a first-dead/second-live backend regression. - P2 (TLS): the pool was built with ssl.create_default_context(), which trusts the system roots rather than the certifi + SSL_CERT_FILE/SSL_CERT_DIR trust the default httpx client uses — chains that verified before could silently stop. Build the context via httpx.create_ssl_context() instead; hostname verification and SNI stay bound to the URL host. Adds coverage for the source.
|
Both fixed, thanks — the fallback one especially would have bitten dual-stack setups. Multi-address fallback. The pin now carries the whole validated set instead of just the first entry ( TLS trust. Swapped Same caveat as yours on validation: I didn't run live external dual-stack/TLS or custom-CA/corporate-proxy paths — the new coverage is the socket-level fallback and the CA-source assertion. |
…nned transport RaresKeY review on odysseus-dev#5514: - P2 (networking): the pin kept only the first resolved IP, so an approved but unreachable first address failed the whole call even when another validated address was healthy. check_outbound_url only returns ok when *every* resolved IP is safe, so the full set is guard-approved — carry all of them and let the pinned backend fall back address-by-address at connect time. Fallback shares one connect deadline (no N*timeout blow-up), never re-resolves, and never replays the request. Adds a first-dead/second-live backend regression. - P2 (TLS): the pool was built with ssl.create_default_context(), which trusts the system roots rather than the certifi + SSL_CERT_FILE/SSL_CERT_DIR trust the default httpx client uses — chains that verified before could silently stop. Build the context via httpx.create_ssl_context() instead; hostname verification and SNI stay bound to the URL host. Adds coverage for the source.
cbddb1c to
7b98515
Compare
|
Small follow-up on the fallback test — swapped the fake-backend version for a real-socket one ( |
Summary
execute_api_call(theapi_callagent tool) validated its outbound URL withcheck_outbound_urlin #5145, but then opened a plainhttpx.AsyncClient()and requested the hostname URL.check_outbound_urlresolves the host to decide accept/reject and returns only(ok, reason)— it hands back no IP — so httpx resolves the host a second time at connect. When those two lookups disagree (DNS rebinding, a low-TTL record that flips public →169.254.169.254), an internal or cloud-metadata address is reached after the guard already approved a public one, with the integration's stored auth headers attached.The repo already closes this exact TOCTOU on every other outbound path by pinning the connection to the address the guard validated: web fetch (#704 / issue #2892) and webhook delivery (#5147 / issue #5146).
api_callgot the validation half but not the pin, so it was the last path still exposed.api_callis agent-reachable (do_api_callinsrc/tools/system.py), so untrusted content driving the agent can trigger it.The fix resolves the host once through the guard's own resolver, remembers the IPs it validated, and pins the request connection to one of them with a small async transport. The URL is passed through unchanged, so TLS SNI and the
Hostheader stay the original hostname — only the socket destination is pinned. LAN integrations (Home Assistant, Miniflux, ntfy) keep working: private/loopback stays allowed by default and is still gated byINTEGRATION_API_BLOCK_PRIVATE_IPS.I kept the pinned transport local to
integrations.pyrather than importing web fetch's or webhook's copy — that mirrors how those two modules each carry their own, and avoids couplingapi_callto the webhook subsystem.Target branch
dev, notmain.Linked Issue
Fixes #5513
Type of Change
Checklist
devdocker compose uporuvicorn app:app) and verified the change works end-to-end. Type-checks and unit tests are not enough.How to Test
pytest tests/test_integration_api_call_ssrf.py tests/test_integrations_api_call_truncation.pytest_connection_is_pinned_to_the_validated_ipfails ondev(a plain client, no pin) and passes here;test_pin_uses_a_validated_ip_when_host_has_severalcovers multi-record hosts.base_urlat a hostname that resolves to a local HTTP server, then flip that hostname's connect-time DNS answer to an unrouteable IP. The request still lands on the validated server (the pin ignores the rebind), and theHostheader + bearer credential arrive unchanged — while a LAN base_url (192.168.x.x) is unaffected by default and blocked whenINTEGRATION_API_BLOCK_PRIVATE_IPS=true.No
static/changes, so no screenshot.