Skip to content

fix(orval): decode escaped JSON Pointer tokens in external $refs (#3380) - #3382

Merged
melloware merged 2 commits into
orval-labs:masterfrom
wadakatu:fix/issue-3380-external-path-ref
May 18, 2026
Merged

fix(orval): decode escaped JSON Pointer tokens in external $refs (#3380)#3382
melloware merged 2 commits into
orval-labs:masterfrom
wadakatu:fix/issue-3380-external-path-ref

Conversation

@wadakatu

@wadakatu wadakatu commented May 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #3380.

A cross-file path-item $ref (e.g. $ref: "common.yaml#/paths/~1pets") aborts generation with an INVALID_REFERENCE validation error. External path-item $ref is valid OpenAPI 3.x, and the same problem affects external $refs into any non-components/schemas location whose JSON Pointer contains escaped characters.

Root cause

@scalar/json-magic bundles external files under x-ext and rewrites refs to #/x-ext/<key>/.... replaceXExtRefs (packages/orval/src/import-specs.ts) then resolves those by splitting the pointer on / and indexing the external document segment by segment.

The segments are used verbatim, without decoding:

  • JSON Pointer escapes — ~1 for /, ~0 for ~ (RFC 6901)
  • Percent-encoding — e.g. %7B/%7D for {/} in templated paths

So the segment ~1pets never matches the real /pets key, the #/x-ext/... ref is left unresolved, and validation rejects it. Because every path key starts with /, every external path-item $ref is affected. Component-schema refs happen to work only because their pointers contain no escaped characters.

This is the external-file counterpart of #398#398 fixed escaped-token resolution in the core resolver (resolveValue, PR #3355), but the x-ext bundling path in import-specs.ts was not covered.

Fix

Add decodeRefToken, applied to each pointer segment before walking the external document: percent-decoding first (outer layer; malformed sequences fall back to the raw token instead of throwing), then JSON Pointer unescaping (~1/, ~0~). The change is scoped to the inline-walk branch, which covers path items and any other non-components/schemas external ref.

Tests

  • Unit (import-specs.test.ts) — dereferenceExternalRef now resolves an x-ext path-item ref containing both ~1 and %7B/%7D. Verified red before the fix, green after.
  • Integration (tests/specifications/issue-3380/) — a spec whose path items are cross-file $refs (~1pets and ~1pets~1%7BpetId%7D); a focused assertion in api-generation.spec.ts confirms both operations are generated, alongside the snapshot.

All checks pass locally: format:check, lint, typecheck, test, test:snapshots, generated typecheck, and mock verification.

Summary by CodeRabbit

  • Bug Fixes

    • Fixed dereferencing of external path-item references with percent-encoded and JSON Pointer-escaped tokens, restoring correct handling of templated paths, inlined operations, and removal of intermediate extension markers after resolution.
  • Tests

    • Added regression tests, fixtures and a generated snapshot asserting cross-file external path refs, encoded/escaped pointer segments, and templated path parameter resolution (includes emitted client endpoints for listPets/getPet).

Review Change Stack

Copilot AI review requested due to automatic review settings May 18, 2026 18:00
@coderabbitai

coderabbitai Bot commented May 18, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 85d176c6-ea82-47da-861d-3e922d695a89

📥 Commits

Reviewing files that changed from the base of the PR and between 49b6cf1 and d172bac.

📒 Files selected for processing (7)
  • packages/orval/src/import-specs.test.ts
  • packages/orval/src/import-specs.ts
  • tests/__snapshots__/default/issue-3380-external-path-ref/endpoints.ts
  • tests/api-generation.spec.ts
  • tests/configs/default.config.ts
  • tests/specifications/issue-3380/issue-3380-common.yaml
  • tests/specifications/issue-3380/issue-3380.yaml
✅ Files skipped from review due to trivial changes (1)
  • tests/snapshots/default/issue-3380-external-path-ref/endpoints.ts
🚧 Files skipped from review as they are similar to previous changes (3)
  • tests/configs/default.config.ts
  • packages/orval/src/import-specs.ts
  • packages/orval/src/import-specs.test.ts

📝 Walkthrough

Walkthrough

Decodes percent-encoded URI fragment tokens and JSON Pointer escapes when resolving external #/x-ext/... $ref segments; adds unit and integration tests, OpenAPI fixtures, config, and a generated snapshot to validate /pets, /pets/{petId}, and /pets~dogs resolution and x-ext removal.

Changes

External Path-Item Reference Decoding

Layer / File(s) Summary
Decode reference token helper and integration
packages/orval/src/import-specs.ts
Added decodeRefToken helper to decode percent-encoded URI fragment tokens and JSON Pointer escapes (~1/, ~0~) with fallback for malformed encoding; integrated into replaceXExtRefs to decode each $ref path segment before traversal.
Unit test for dereferenceExternalRef with escaped tokens
packages/orval/src/import-specs.test.ts
New test verifies dereferenceExternalRef resolves external path-item $ref entries with escaped JSON Pointer tokens and percent-encoded templated segments, checks inlined get operations and petId parameter, and confirms x-ext removal.
OpenAPI test specifications for issue #3380
tests/specifications/issue-3380/issue-3380.yaml, tests/specifications/issue-3380/issue-3380-common.yaml
Main spec uses external $ref targets with JSON Pointer escapes and percent-encoding; common file provides the referenced /pets and /pets/{petId} path-item definitions.
Configuration, snapshot, and integration test
tests/configs/default.config.ts, tests/__snapshots__/default/issue-3380-external-path-ref/endpoints.ts, tests/api-generation.spec.ts
Adds Orval config entry for the fixture, a generated endpoints snapshot exporting listPets and getPet, and an integration test that asserts the templated path decodes to /pets/${petId}.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • orval-labs/orval#3336: Related changes to replaceXExtRefs in import-specs.ts (cycle prevention) that touch the same external $ref inlining logic.

Suggested labels

bug

Suggested reviewers

  • melloware
  • soartec-lab

Poem

🐰 I hopped through tokens, percent and tilde too,
~1 turned to slash, ~0 became true,
Paths once hidden now step into light,
/pets and {petId} stitched back tight,
A rabbit's cheer for refs made right.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: fixing decoding of escaped JSON Pointer tokens in external $refs, which is the core issue addressed throughout the pull request.
Linked Issues check ✅ Passed The PR fully addresses issue #3380 by implementing the required fix: decoding percent-encoded and JSON Pointer-escaped tokens in external path-item $refs through the new decodeRefToken helper function and comprehensive test coverage.
Out of Scope Changes check ✅ Passed All changes are directly scoped to resolving issue #3380: the core fix in import-specs.ts, corresponding unit tests, integration test fixtures, and snapshot generation are all necessary and related to the issue.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
packages/orval/src/import-specs.ts (1)

347-357: 💤 Low value

Consider adding JSDoc to document the decoding order.

The inline comment is helpful, but a JSDoc block would make the two-layer decoding contract (percent-encoding outer, JSON Pointer inner) more discoverable for future maintainers.

📝 Optional JSDoc addition
+/**
+ * Decode a single JSON Pointer reference token taken from an x-ext `$ref`.
+ *
+ * The token carries two layers of encoding: it sits in a URI fragment, so it
+ * may be percent-encoded (e.g. `%7B` for `{` in templated paths), and it is a
+ * JSON Pointer token, so `~1`/`~0` stand for `/`/`~` (RFC 6901). Percent-
+ * encoding is the outer layer and is removed first; a malformed sequence is
+ * left as-is rather than throwing. Without this, tokens such as `~1pets`
+ * never match the real `/pets` key and the external `$ref` fails to resolve.
+ *
+ * `@param` token - Raw reference token from a split `$ref` path
+ * `@returns` Decoded token ready for object property lookup
+ */
 function decodeRefToken(token: string): string {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/orval/src/import-specs.ts` around lines 347 - 357, Add a JSDoc block
above the decodeRefToken function that documents the two-layer decoding
contract: percent-decoding (URI fragment) is applied first and then JSON Pointer
unescaping (~1 → /, ~0 → ~), note that malformed percent-encodings are left
unchanged (no throw), and include an example showing a token like "%7B~1pets" ->
"{/pets" to make the decoding order and behavior explicit for future
maintainers; reference the function name decodeRefToken in the JSDoc.
packages/orval/src/import-specs.test.ts (1)

764-801: 💤 Low value

Consider adding test coverage for ~0 (tilde escape).

The test validates ~1 (slash) and percent-encoding (%7B/%7D), but doesn't explicitly test ~0~ decoding. While the implementation is straightforward, adding a path like /pets~0dogs (for /pets~dogs) would complete the RFC 6901 coverage.

🧪 Optional test case addition

Add a third path to the test input:

         '/pets/{petId}': {
           $ref: '`#/x-ext/abc1234/paths/`~1pets~1%7BpetId%7D',
         },
+        '/pets~dogs': {
+          $ref: '`#/x-ext/abc1234/paths/`~1pets~0dogs',
+        },
       },

And corresponding external definition plus assertion:

             '/pets/{petId}': {
               get: {
                 operationId: 'getPet',
                 parameters: [
                   {
                     name: 'petId',
                     in: 'path',
                     required: true,
                     schema: { type: 'string' },
                   },
                 ],
                 responses: { '200': { description: 'ok' } },
               },
             },
+            '/pets~dogs': {
+              get: {
+                operationId: 'listPetsDogs',
+                responses: { '200': { description: 'ok' } },
+              },
+            },
           },
         },
       },
     expect(result.paths?.['/pets/{petId}']).toEqual({
       get: {
         operationId: 'getPet',
         parameters: [
           {
             name: 'petId',
             in: 'path',
             required: true,
             schema: { type: 'string' },
           },
         ],
         responses: { '200': { description: 'ok' } },
       },
     });
+    expect(result.paths?.['/pets~dogs']).toEqual({
+      get: {
+        operationId: 'listPetsDogs',
+        responses: { '200': { description: 'ok' } },
+      },
+    });
     expect(result).not.toHaveProperty('x-ext');
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/orval/src/import-specs.test.ts` around lines 764 - 801, Add a third
path to the test input to verify RFC6901 tilde-unescape: add an entry in the
top-level input.paths with the key '/pets~0dogs' pointing via $ref to the
external definition (use the JSON Pointer-escaped path under x-ext, e.g.
'`#/x-ext/abc1234/paths/`~1pets~00dogs'), add the matching definition under
input['x-ext'].abc1234.paths with an operation (e.g. get with operationId
'listPetsTilde') and then add an assertion that the resolved spec contains the
decoded path '/pets~dogs' (or that the operationId is present for '/pets~dogs')
to ensure '~0' is decoded to '~'.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/orval/src/import-specs.test.ts`:
- Around line 764-801: Add a third path to the test input to verify RFC6901
tilde-unescape: add an entry in the top-level input.paths with the key
'/pets~0dogs' pointing via $ref to the external definition (use the JSON
Pointer-escaped path under x-ext, e.g. '`#/x-ext/abc1234/paths/`~1pets~00dogs'),
add the matching definition under input['x-ext'].abc1234.paths with an operation
(e.g. get with operationId 'listPetsTilde') and then add an assertion that the
resolved spec contains the decoded path '/pets~dogs' (or that the operationId is
present for '/pets~dogs') to ensure '~0' is decoded to '~'.

In `@packages/orval/src/import-specs.ts`:
- Around line 347-357: Add a JSDoc block above the decodeRefToken function that
documents the two-layer decoding contract: percent-decoding (URI fragment) is
applied first and then JSON Pointer unescaping (~1 → /, ~0 → ~), note that
malformed percent-encodings are left unchanged (no throw), and include an
example showing a token like "%7B~1pets" -> "{/pets" to make the decoding order
and behavior explicit for future maintainers; reference the function name
decodeRefToken in the JSDoc.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: fb77f9d4-dfd0-4541-9e17-fb8008d775ba

📥 Commits

Reviewing files that changed from the base of the PR and between b978e04 and 6751077.

📒 Files selected for processing (7)
  • packages/orval/src/import-specs.test.ts
  • packages/orval/src/import-specs.ts
  • tests/__snapshots__/default/issue-3380-external-path-ref/endpoints.ts
  • tests/api-generation.spec.ts
  • tests/configs/default.config.ts
  • tests/specifications/issue-3380/issue-3380-common.yaml
  • tests/specifications/issue-3380/issue-3380.yaml

@melloware melloware added the openapi OpenAPI related issue label May 18, 2026
@wadakatu

Copy link
Copy Markdown
Contributor Author

Thanks for the review! Addressed the two nitpicks:

  • ~0 test coverage — added a /pets~dogs case (~1pets~0dogs) to the unit test in 49b6cf1, so the test now exercises all three escapes the helper handles: ~1/, ~0~, and percent-encoding.
  • JSDoc on decodeRefToken — leaving as-is. decodeRefToken already carries a JSDoc block documenting the two-layer decoding contract (percent-encoding outer, JSON Pointer inner) and the malformed-input fallback. Adding @param/@returns tags would be inconsistent with the sibling functions in import-specs.ts (scrubUnwantedKeys, updateInternalRefs, replaceXExtRefs, dereferenceExternalRef), which all use prose-only JSDoc.

@wadakatu
wadakatu force-pushed the fix/issue-3380-external-path-ref branch from 49b6cf1 to d172bac Compare May 18, 2026 18:25
@melloware
melloware merged commit cd010a4 into orval-labs:master May 18, 2026
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

openapi OpenAPI related issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

External path-item $ref fails with "Can't resolve reference" (JSON Pointer escapes not decoded)

2 participants