Skip to content

[TT-16890] fix: backport #7974 — validate middleware collapsed path fix#8067

Merged
buger merged 2 commits intorelease-5.12.1from
backport/7974-to-5.12.1
Apr 17, 2026
Merged

[TT-16890] fix: backport #7974 — validate middleware collapsed path fix#8067
buger merged 2 commits intorelease-5.12.1from
backport/7974-to-5.12.1

Conversation

@buger
Copy link
Copy Markdown
Member

@buger buger commented Apr 17, 2026

Summary

Clean backport of #7974 to release-5.12.1. Replaces #8060 which had cherry-pick issues (empty diff, 0 changed files).

  • Adds collapsed parameterized path disambiguation for validate-request and mock-response OAS middleware
  • Cherry-picked from merge commit d89cf0b with conflict resolution for the 2-value mockResponse return signature on this branch

Test plan

  • Unit Tests & Linting passes
  • CI Tests pass

🤖 Generated with Claude Code

Backport of [TT-16890] from master to release-5.12.1.
Adds collapsed parameterized path disambiguation for validate-request
and mock-response OAS middleware.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@probelabs
Copy link
Copy Markdown
Contributor

probelabs bot commented Apr 17, 2026

This pull request backports a fix from #7974 to resolve ambiguity when multiple OpenAPI parameterized paths compile to the same regular expression (e.g., /users/{id:integer} and /users/{name:string}). Previously, this "collapsed path" scenario could lead to incorrect validation rules or mock responses being applied. The fix introduces a disambiguation mechanism that correctly routes requests based on path parameter schemas.

Files Changed Analysis

The changes are primarily focused on the gateway's OpenAPI processing logic and are supported by a substantial addition of test cases:

  • gateway/api_definition.go: Implements the core logic for detecting and grouping collapsed paths during API definition loading. It introduces a scoring system to sort these ambiguous paths (candidates) based on the restrictiveness of their parameter schemas (e.g., integer is more restrictive than string).
  • gateway/model_urlspec.go & gateway/model_apispec.go: The URLSpec data structure is updated to store the sorted candidates. A shared helper method, matchCandidatePath, is added to APISpec for schema validation.
  • gateway/mw_oas_validate_request.go: The validation middleware is enhanced to handle these candidates. When a request matches an ambiguous path, it iterates through the sorted candidates and uses the first one whose parameter schema matches the request path for full validation.
  • gateway/mw_mock_response.go: The mock response middleware is updated with the same disambiguation logic to ensure the correct mock response is selected.
  • gateway/*_test.go: Two new test files (mw_mock_response_collapsed_test.go, mw_oas_validate_request_scenarios_test.go) and significant additions to an existing one add over 3,000 lines of tests, thoroughly validating the new prioritization logic across numerous scenarios.

Architecture & Impact Assessment

  • What this PR accomplishes: It resolves a critical issue in the OAS middleware, ensuring that requests to overlapping parameterized paths are validated and mocked correctly. This allows for more intuitive and standard REST API designs, such as identifying a resource by either a numeric ID or a string slug.

  • Key technical changes introduced:

    1. Load-Time Path Grouping: At API load time, paths that generate identical regular expressions are grouped into a list of candidates.
    2. Schema-Based Sorting: These candidates are sorted using a scoring system that prioritizes stricter schemas (e.g., integer > number > string with a pattern > unconstrained string).
    3. Request-Time Disambiguation: At request time, the middleware performs a two-phase check. It first validates the request's path parameters against the sorted candidates' schemas. The first successful match determines which endpoint's rules are used for the full request validation or mock response.
  • Affected system components: The changes are localized to the Gateway's OAS Middleware Engine, specifically affecting the validate-request and mock-response functionalities.

Request Disambiguation Flow

sequenceDiagram
    participant Client
    participant Gateway
    participant APILoader as API Definition Loader
    participant Middleware

    Note over APILoader: On API Load
    APILoader->>APILoader: 1. Compile OAS paths to regex
    APILoader->>APILoader: 2. Group paths with same regex (e.g., /users/{id}, /users/{name})
    APILoader->>APILoader: 3. Score & sort candidates by schema restrictiveness

    Note over Gateway: On Request
    Client->>Gateway: GET /api/users/123
    Gateway->>Middleware: Process request
    Middleware->>Middleware: Find URLSpec with candidates
    Middleware->>Middleware: Test "123" against {id:integer} schema -> Match!
    Middleware->>Middleware: Commit to {id:integer} endpoint for full validation
    Middleware-->>Gateway: Validation result
    Gateway-->>Client: Response
Loading

Scope Discovery & Context Expansion

The scope of this PR is well-contained within the Tyk Gateway's handling of OpenAPI specifications. The changes do not impact non-OAS APIs or other middleware functionalities.

  • The fix is crucial for developers who rely on the validate-request and mock-response middleware and use common API design patterns with multiple identifiers for a resource.
  • The files modified are appropriate and comprehensive for this change. The logic is correctly placed at the API loading stage (api_definition.go), data structures are updated accordingly (model_*.go), and the relevant middleware (mw_*.go) are modified to use the new mechanism. The extensive test coverage indicates a low risk of regression and a thorough implementation.
Metadata
  • Review Effort: 4 / 5
  • Primary Label: bug

Powered by Visor from Probelabs

Last updated: 2026-04-17T05:54:28.357Z | Triggered by: pr_updated | Commit: e4c1b04

💡 TIP: You can chat with Visor using /visor ask <your question>

@probelabs
Copy link
Copy Markdown
Contributor

probelabs bot commented Apr 17, 2026

Security Issues (1)

Severity Location Issue
🟡 Warning gateway/mw_oas_validate_request.go:194
The custom UUID validation regex `^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$` is used for path parameter validation when the OpenAPI format is 'uuid'. This regex is potentially inefficient and could be vulnerable to ReDoS (Regular Expression Denial of Service) if a malicious, long, non-matching string is provided as a path parameter. The standard library's `regexp` package does not have backtracking control mechanisms, making it susceptible to catastrophic backtracking with certain patterns and inputs.
💡 SuggestionReplace the custom regex validation for UUIDs with a call to a standard, optimized UUID parsing function like `uuid.Parse()` from a reputable library (e.g., `github.com/google/uuid`). This avoids the potential for ReDoS and ensures compliance with the UUID standard.

Architecture Issues (2)

Severity Location Issue
🟡 Warning gateway/api_definition.go:87-141
The functions `mergeMockGroupIntoPrimary` and `mergeGroupIntoPrimary` are nearly identical, with the only significant difference being the candidate struct type being created (`MockResponseCandidate` vs `ValidateRequestCandidate`). This creates code duplication that could be eliminated.
💡 SuggestionRefactor these two functions into a single, generic function that accepts the candidate type or a factory function as an argument. This would reduce code duplication and improve maintainability, as any future logic changes would only need to be applied in one place.
🟡 Warning gateway/mw_oas_validate_request.go:217-279
The function `valueMatchesSchema` and its helper `valueMatchesFormat` re-implement a subset of the validation logic that exists within the `kin-openapi` library. While this is a pragmatic necessity for the pre-filtering step to disambiguate paths, it introduces a maintenance risk. The behavior of this custom validation logic could diverge from the underlying library over time, leading to subtle bugs where the pre-filtering decision does not match the result of the full validation.
💡 SuggestionTo mitigate the risk of divergence, add a prominent comment to the `valueMatchesSchema` function explaining that it is a partial re-implementation of `kin-openapi`'s path parameter validation. This comment should advise future developers to ensure this function remains synchronized with the behavior of the `openapi3filter.ValidateRequest` function from the library, especially when upgrading `kin-openapi`.

Security Issues (1)

Severity Location Issue
🟡 Warning gateway/mw_oas_validate_request.go:194
The custom UUID validation regex `^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$` is used for path parameter validation when the OpenAPI format is 'uuid'. This regex is potentially inefficient and could be vulnerable to ReDoS (Regular Expression Denial of Service) if a malicious, long, non-matching string is provided as a path parameter. The standard library's `regexp` package does not have backtracking control mechanisms, making it susceptible to catastrophic backtracking with certain patterns and inputs.
💡 SuggestionReplace the custom regex validation for UUIDs with a call to a standard, optimized UUID parsing function like `uuid.Parse()` from a reputable library (e.g., `github.com/google/uuid`). This avoids the potential for ReDoS and ensures compliance with the UUID standard.
\n\n ### Architecture Issues (2)
Severity Location Issue
🟡 Warning gateway/api_definition.go:87-141
The functions `mergeMockGroupIntoPrimary` and `mergeGroupIntoPrimary` are nearly identical, with the only significant difference being the candidate struct type being created (`MockResponseCandidate` vs `ValidateRequestCandidate`). This creates code duplication that could be eliminated.
💡 SuggestionRefactor these two functions into a single, generic function that accepts the candidate type or a factory function as an argument. This would reduce code duplication and improve maintainability, as any future logic changes would only need to be applied in one place.
🟡 Warning gateway/mw_oas_validate_request.go:217-279
The function `valueMatchesSchema` and its helper `valueMatchesFormat` re-implement a subset of the validation logic that exists within the `kin-openapi` library. While this is a pragmatic necessity for the pre-filtering step to disambiguate paths, it introduces a maintenance risk. The behavior of this custom validation logic could diverge from the underlying library over time, leading to subtle bugs where the pre-filtering decision does not match the result of the full validation.
💡 SuggestionTo mitigate the risk of divergence, add a prominent comment to the `valueMatchesSchema` function explaining that it is a partial re-implementation of `kin-openapi`'s path parameter validation. This comment should advise future developers to ensure this function remains synchronized with the behavior of the `openapi3filter.ValidateRequest` function from the library, especially when upgrading `kin-openapi`.
\n\n ### Performance Issues (1)
Severity Location Issue
🟡 Warning gateway/mw_oas_validate_request.go:162
The enum validation within `valueMatchesSchema` uses `fmt.Sprintf("%v", e)` inside a loop over enum values. This is inefficient as it relies on reflection and causes a new string allocation for every enum value on every request. For path parameters with large enums, this can introduce noticeable latency in a request hot path.
💡 SuggestionAvoid using `fmt.Sprintf` inside this per-request loop. Consider using a type switch on the enum value `e` to perform a more direct and efficient string conversion for common types (string, int, float). Alternatively, for even better performance, the enum values could be pre-converted to a `map[string]struct{}` at API load time, allowing for a fast lookup instead of a loop.

Powered by Visor from Probelabs

Last updated: 2026-04-17T05:54:11.261Z | Triggered by: pr_updated | Commit: e4c1b04

💡 TIP: You can chat with Visor using /visor ask <your question>

pkg/schema was introduced on master by PR #7904 and is not available
on release branches. Replace schema.RestoreUnicodeEscapesInError(err)
with plain err to match existing release branch error handling.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

API Changes

--- prev.txt	2026-04-17 05:53:26.485322992 +0000
+++ current.txt	2026-04-17 05:53:16.282363500 +0000
@@ -11268,6 +11268,14 @@
 
 func (m *MockReadCloser) Read(p []byte) (n int, err error)
 
+type MockResponseCandidate struct {
+	OASMockResponseMeta *oas.MockResponse
+	OASMethod           string
+	OASPath             string
+}
+    MockResponseCandidate represents one OAS endpoint that maps to the same
+    compiled regex pattern for mock response disambiguation.
+
 type Monitor struct {
 	Gw *Gateway `json:"-"`
 }
@@ -12644,6 +12652,16 @@
 	OASValidateRequestMeta    *oas.ValidateRequest
 	OASMockResponseMeta       *oas.MockResponse
 
+	// OASValidateRequestCandidates holds multiple OAS endpoints that compile to the
+	// same regex pattern. When non-empty, the validate request middleware must
+	// disambiguate by checking path parameter schemas against each candidate.
+	OASValidateRequestCandidates []ValidateRequestCandidate
+
+	// OASMockResponseCandidates holds multiple OAS endpoints that compile to the
+	// same regex pattern. When non-empty, the mock response middleware must
+	// disambiguate by checking path parameter schemas against each candidate.
+	OASMockResponseCandidates []MockResponseCandidate
+
 	IgnoreCase bool
 	// OASMethod stores the HTTP method for OAS-specific middleware
 	// This is needed because OAS operations are method-specific
@@ -12746,6 +12764,16 @@
     ProcessRequest will run any checks on the request on the way through the
     system, return an error to have the chain fail
 
+type ValidateRequestCandidate struct {
+	OASValidateRequestMeta *oas.ValidateRequest
+	OASMethod              string
+	OASPath                string
+}
+    ValidateRequestCandidate represents one OAS endpoint that maps to the
+    same compiled regex pattern. Used for disambiguation when multiple
+    parameterized paths collapse to the same regex (e.g., /employees/{prct} and
+    /employees/{zd}).
+
 type ValueExtractor struct {
 	BaseExtractor
 }

@buger buger merged commit 85f5a59 into release-5.12.1 Apr 17, 2026
11 of 13 checks passed
@buger buger deleted the backport/7974-to-5.12.1 branch April 17, 2026 05:55
buger added a commit that referenced this pull request Apr 17, 2026
Cherry-pick critical fixes that were on release-5.12.1 but missing
from release-5.12:
- #8029: [TT-16890] validate request middleware regression fix
- #8067: backport #7974 validate middleware collapsed path fix
- #7862: dependency updates in go.mod and go.sum

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
buger added a commit that referenced this pull request Apr 17, 2026
## Summary
Sync critical commits that are on release-5.12.1 but missing from
release-5.12.

The branches diverged on Feb 16 with no merge flow between them. This PR
brings release-5.12 up to parity with release-5.12.1 for critical fixes.

## Missing commits synced
- #8029: [TT-16890] validate request middleware regression fix
(CRITICAL)
- #8067: backport #7974 validate middleware collapsed path fix
- #7862: dependency updates in go.mod and go.sum

## Test plan
- [ ] Unit Tests & Linting passes
- [ ] go build ./gateway/... passes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

[TT-16890]:
https://tyktech.atlassian.net/browse/TT-16890?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@probelabs probelabs bot changed the title fix: backport #7974 — validate middleware collapsed path fix [TT-16890] fix: backport #7974 — validate middleware collapsed path fix Apr 17, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🚨 Jira Linter Failed

Commit: e4c1b04
Failed at: 2026-04-17 16:48:40 UTC

The Jira linter failed to validate your PR. Please check the error details below:

🔍 Click to view error details
failed to get Jira issue: failed to fetch Jira issue TT-16890: Issue does not exist or you do not have permission to see it.: request failed. Please analyze the request body for more details. Status code: 404

Next Steps

  • Ensure your branch name contains a valid Jira ticket ID (e.g., ABC-123)
  • Verify your PR title matches the branch's Jira ticket ID
  • Check that the Jira ticket exists and is accessible

This comment will be automatically deleted once the linter passes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant