Preserve the response when a redirect has an invalid Location and redirects are not followed

[devl00p]

Description

When follow_redirects=False and a 3XX response carries a malformed Location header (e.g. a data: URI), _send_handling_redirects builds the redirect request before checking follow_redirects, so _redirect_url raises RemoteProtocolError and the response — along with its headers — is discarded. A caller that only wants to inspect the redirect without following it never gets the response.

This is a real-world need: the Wapiti web scanner reads the Location header of such responses to detect reflected payloads, and currently cannot — see wapiti-scanner/wapiti#690. A malformed Location is still part of an otherwise valid HTTP response; not following the redirect shouldn't mean losing the response.

The same fix was proposed upstream in encode/httpx#3706 (still open, refs discussion encode/httpx#3179) and is already shipped by the httpxyz fork.

The fix

Defer building the redirect request to the branch that actually follows redirects. When not following, building next_request is best-effort: a malformed Location leaves next_request=None and the response is returned intact. Applied to both the sync and async _send_handling_redirects.

response = httpx2.get(url, follow_redirects=False)  # url 302s to a data: URI
assert response.status_code == 302
assert response.headers["location"] == "data:..."   # header preserved
assert response.next_request is None

Following redirects (follow_redirects=True) still raises RemoteProtocolError on an invalid Location, unchanged.

Tests

Adds sync and async regression tests (test_invalid_redirect_not_followed) reusing the existing /invalid_redirect mock endpoint. scripts/check (ruff format, mypy, ruff check, unasync) and the redirect test suite pass.

[codspeed-hq]

Merging this PR will not alter performance

✅ 15 untouched benchmarks
⏩ 7 skipped benchmarks¹

---

_{Comparing devl00p:fix/preserve-response-on-invalid-redirect-location (3f56208) with main (0dd0ee0)}

Footnotes

1. 7 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[cubic-dev-ai]

No issues found across 2 files

_{Re-trigger cubic}