fix(rewrites): preserve external destination path prefixes#2555
fix(rewrites): preserve external destination path prefixes#2555james-elicx wants to merge 4 commits into
Conversation
|
/bigbonk review for issues |
commit: |
|
Review: fix(rewrites): preserve external destination path prefixesI traced the full request pipeline for every runtime touched by this PR. The 1. (Blocking) The added unit tests assert behavior that cannot occur in the real pipeline — single-encoded
|
|
Review posted to PR #2555. Summary of findings: Review SummaryI traced the full request pipeline across all runtimes touched by the PR. The destination-encoding change itself is sound, but I found a test/pipeline parity gap: Blocking
Non-blocking
The core recommendation: reconcile the unit-test inputs with |
|
/bigbonk review for issues |
Performance benchmarksCompared 0 improved · 0 regressed · 6 within ±1.5%
View detailed results and traces 🟢 improvement · 🔴 regression · ⚫ change below 1.5% · paired base/head |
There was a problem hiding this comment.
Review: fix(rewrites): preserve external destination path prefixes
I independently re-traced the full request pipeline across dev, Node prod (App + Pages), and Cloudflare Workers (App + Pages). The destination-encoding change (substituteDestinationParams gaining path-aware param encoding via encodeDestinationPathParamValue/encodeDestinationPathSegment) is sound and is what actually produces the correct external-rewrite output. I confirm the two blocking issues already raised, and add one more concrete finding that strengthens them.
1. (Blocking — confirmed) Unit tests assert behavior that cannot reach matchRewrite in the real pipeline
tests/shims.test.ts calls matchRewrite("/proxy/%2e%2e/outside", …) (single-encoded) and asserts …/v1/%252e%252e/outside, and escapeUrlDotSegments("/proxy/.%2e/outside") → /proxy/.%252e/outside. Neither input can reach those functions in production. In every entry (index.ts:4731, prod-server.ts:1656, app-rsc-request-normalization.ts:96-102), normalizePath(normalizePathnameForRouteMatchStrict(pathname)) runs first and:
- decodes each segment (
%2e%2e→..,.%2e→..,%2e.→..) - resolves
./..(/proxy/../outside→/outside)
I reproduced the pipeline faithfully:
/proxy/%2e%2e/outside -> normalize -> /outside
/proxy/.%2e/outside -> normalize -> /outside
/proxy/%2e./outside -> normalize -> /outside
/proxy/%252e%252e/... -> normalize -> /proxy/%2e%2e/... (double-encoded survives)
Every single-encoded case collapses to /outside, so it never matches /proxy/:path* and escapeUrlDotSegments is a no-op for exactly the inputs the unit test claims to cover. The E2E/integration specs pass only because they use double-encoded %252e%252e. Unit and E2E coverage are therefore testing incompatible inputs, and the unit test gives false confidence about single-encoded traversal preservation.
Fix: change the unit tests to assert the double-encoded values that actually reach matchRewrite/escapeUrlDotSegments in each runtime, so unit and E2E coverage agree. (Or, if single-encoded preservation is genuinely intended, move the escape ahead of normalizePath — but that changes traversal-normalization semantics and must be verified against Next.js first.)
2. (Blocking — confirmed, with new evidence) The runtimes take structurally different routes to the same output; escapeUrlDotSegments is redundant or dead in most of them
escapeUrlDotSegments is only used in index.ts:4742 (dev Pages) and prod-server.ts:1442/:1663. I traced where the escape actually influences the value matchRewrite sees:
| Runtime | Pathname reaching matchRewrite (for %252e%252e) |
Escaping performed by |
|---|---|---|
| Pages prod (Node) | %252e%252e |
escapeUrlDotSegments (pipeline reads new URL().pathname w/o re-normalize) |
| Pages dev (Node) | %252e%252e |
escapeUrlDotSegments |
| App Router prod (Node) | %2e%2e |
encodeDestinationPathSegment — the escapeUrlDotSegments at prod-server.ts:1442 is undone by the RSC handler's re-normalization (app-rsc-request-normalization.ts runs normalizePath again on request.url, collapsing %252e%252e back to %2e%2e) |
| CF Workers (App + Pages) | %2e%2e |
encodeDestinationPathSegment — escapeUrlDotSegments is never called; pages-router-entry.ts:112-113 uses new URL(request.url).pathname directly and dev short-circuits at index.ts:4788 (if (hasCloudflarePlugin) return next();) |
I verified the App-Router-prod round-trip:
prod-server pathname: /proxy/%2e%2e/outside
escapeUrlDotSegments -> request.url: /proxy/%252e%252e/outside
RSC re-normalize -> matchRewrite: /proxy/%2e%2e/outside <-- escape undone
So the final output (%252e%252e) is correct everywhere, but it's produced by two independent mechanisms that happen to agree. encodeDestinationPathSegment is doing the real work in App Router (both runtimes) and CF Workers; escapeUrlDotSegments only matters for Node Pages dev/prod. The PR's stated goal ("keep encoded dot segments stable across Node and Workers") is only coincidentally realized. Recommend either unifying on a single mechanism (encodeDestinationPathSegment already covers all runtimes) or documenting the split explicitly — otherwise a future refactor dropping escapeUrlDotSegments from the call sites would leave every E2E green while changing nothing it appears to guard, and the App Router prod escape is already effectively dead.
3. (Non-blocking — confirmed) The two helpers diverge on literal-dot segments
normalize-path.ts:50(escapeUrlDotSegments):./..stay literal,%2e→%252e.config-matchers.ts:1199(encodeDestinationPathSegment):./..→%252e,%2e→%252e.
"." escapeUrlDotSegments: "." encodeDestPathSeg: "%252e"
".." escapeUrlDotSegments: ".." encodeDestPathSeg: "%252e%252e"
".%2e" escapeUrlDotSegments: ".%252e" encodeDestPathSeg: "%252e%252e"
This is latent (literal .. never survives normalizePath to reach either helper), but it's a second reason the two suites can't both be right about literal/mixed-dot inputs. If both helpers are meant to "make dot segments stable," share one implementation and one encoding convention.
4. (Nit — confirmed) playwright.config.ts triple-repeats TEST_EXTERNAL_REWRITE_ORIGIN
The pages-router-prod command chains the same env var across build && build && start. Playwright webServer supports an env: field that applies to the whole command string; hoisting it removes three copies that can drift (they already use four distinct ports 4228–4231).
5. (Info) CodeQL "incomplete string escaping" at config-matchers.ts:1201 is a false positive
CodeQL flags dot.replace("%", "%25") for replacing only the first %. Here dot is a single match of /\.|%2e/gi, so it is either . (no %) or %2e (exactly one %). replace("%", "%25") is sufficient. Consider replaceAll or an inline // codeql[js/incomplete-sanitization] suppression with a rationale to keep the scanner quiet, but there is no correctness bug.
What I verified
- Added unit tests pass:
vp test run tests/shims.test.ts -t "escapeUrlDotSegments|matchRewrite with external URLs"→ 18 passed. - Reproduced
normalizePathnameForRouteMatchStrict+normalizePathcollapsing all single-encoded dot inputs to/outside. - Traced the App-Router-prod re-normalization that undoes
escapeUrlDotSegments, and confirmed CF Workers (pages-router-entry.ts,app-rsc-request-normalization.ts) never call it. - Confirmed every E2E/integration spec added by this PR uses double-encoded
%252e%252e, matching the real pipeline.
Bottom line: the encoding change is correct and safe to ship functionally. Before merge, reconcile the unit-test inputs with normalizePath's actual behavior (use double-encoded inputs), and decide whether escapeUrlDotSegments should stay (and be applied consistently, including CF/App Router) or be dropped in favor of encodeDestinationPathSegment alone — right now it's redundant in 3 of the 4 production runtimes.
|
Review posted to PR #2555. SummaryI independently re-traced the full request pipeline across all runtimes and confirmed all findings from the prior review, plus uncovered additional concrete evidence. Blocking
Non-blocking
The encoding change itself is correct and functionally safe to ship. The recommendation is to reconcile unit-test inputs with the real pipeline (double-encoded) and decide whether |
bb38bab to
b43709a
Compare
|
/bigbonk review for issues |
|
@james-elicx Bonk workflow was cancelled. View workflow run · To retry, trigger Bonk again. |
|
/bigbonk review for issues |
Review: fix(rewrites): preserve external destination path prefixesI independently re-traced the full request pipeline for every runtime this PR touches (Pages dev, Pages Node prod, App Router prod, CF Workers Pages + App). The core encoding change is correct and functionally sound, and importantly, my trace contradicts several claims in the earlier
|
|
Review posted to PR #2555: #2555 (comment) SummaryI independently traced the full request pipeline across all five runtimes and reached conclusions that correct several claims in the prior Verdict: The fix is correct and safe to ship. No blocking issues. Key findings:
Minor, non-blocking suggestions:
Stale prior-review points that don't apply to this diff: the "two diverging helpers" (only one helper exists now) and the playwright |
Summary
Tests
vp check packages/vinext/src/config/config-matchers.ts packages/vinext/src/server/normalize-path.ts packages/vinext/src/index.ts packages/vinext/src/server/prod-server.ts playwright.config.ts tests/shims.test.ts tests/app-router-production-server.test.ts tests/e2e/app-router/route-config.spec.ts tests/e2e/pages-router/config.spec.ts tests/e2e/pages-router-prod/production.spec.ts tests/e2e/cloudflare-pages-router/external-rewrite.spec.ts tests/fixtures/app-basic/next.config.ts tests/fixtures/pages-basic/next.config.mjs examples/pages-router-cloudflare/next.config.mjsvp test run tests/shims.test.ts -t "escapeUrlDotSegments|matchRewrite with external URLs"vp test run tests/app-router-production-server.test.ts -t "preserves external rewrite path prefixes"PLAYWRIGHT_PROJECT=app-router npx playwright test tests/e2e/app-router/route-config.spec.ts --project=app-router --workers=1PLAYWRIGHT_PROJECT=pages-router npx playwright test tests/e2e/pages-router/config.spec.ts --project=pages-router --workers=1PLAYWRIGHT_PROJECT=pages-router-prod npx playwright test tests/e2e/pages-router-prod/production.spec.ts --project=pages-router-prod --workers=1 -g "external catch-all"CI=1 PLAYWRIGHT_PROJECT=cloudflare-pages-router npx playwright test tests/e2e/cloudflare-pages-router/external-rewrite.spec.ts --project=cloudflare-pages-router --workers=1