[TT-12238] Javascript regex issue on OAS API#7904
Conversation
|
This PR addresses an issue with Javascript-style Unicode regex patterns in OpenAPI Specification (OAS) APIs. The previous implementation performed a raw byte replacement on the entire JSON definition, which was brittle and could lead to data corruption. This has been replaced with a robust Visitor pattern that safely traverses the parsed OAS object model, ensuring transformations are applied correctly and only to the Files Changed AnalysisThe changes reflect a clear refactoring of functionality from a generic utility package to a more specialized, schema-aware package:
Architecture & Impact Assessment
graph TD
subgraph "Old Implementation (Unsafe)"
A[POST /apis/oas] --> B{Read Raw JSON Body};
B --> C{Replace `\\uXXXX` in entire byte slice};
C --> D{Parse Modified JSON};
end
subgraph "New Implementation (Robust)"
E[POST /apis/oas] --> F{Read Raw JSON Body};
F --> G{Parse JSON into OAS Struct};
G --> H{Use Visitor to traverse OAS struct};
H --> I{Modify only `pattern` fields};
end
style C fill:#f9f,stroke:#333,stroke-width:2px
style I fill:#9cf,stroke:#333,stroke-width:2px
Scope Discovery & Context ExpansionThe scope of this change is confined to the handling of OAS API definitions within the gateway, specifically addressing the incompatibility between the regex flavor specified by the OAS standard (ECMAScript) and the RE2 engine used by Go.
Metadata
Powered by Visor from Probelabs Last updated: 2026-03-23T07:21:46.865Z | Triggered by: pr_updated | Commit: 69fb5f2 💡 TIP: You can chat with Visor using |
✅ Security Check PassedNo security issues found – changes LGTM. ✅ Architecture Check PassedNo architecture issues found – changes LGTM. ✅ Performance Check PassedNo performance issues found – changes LGTM. ✅ Quality Check PassedNo quality issues found – changes LGTM. Powered by Visor from Probelabs Last updated: 2026-03-23T07:21:38.667Z | Triggered by: pr_updated | Commit: 69fb5f2 💡 TIP: You can chat with Visor using |
|
API Changes --- prev.txt 2026-03-23 07:20:40.815577638 +0000
+++ current.txt 2026-03-23 07:20:32.814605880 +0000
@@ -13238,13 +13238,6 @@
the version definition and ensures that required fields have appropriate
values.
-func RestoreUnicodeEscapesInError(err error) error
- RestoreUnicodeEscapesInError takes an error and applies the
- RestoreUnicodeEscapesInRegexp transformation to its message. For example,
- it converts RE2-compatible escapes like `\x{0041}` back to `\u0041`. It
- returns a new error with the transformed message. If the input error is nil,
- it returns nil.
-
func SetAsDefault(versionName string) option.Option[apidef.VersionDefinition]
SetAsDefault creates an option that marks a specific version as the default.
This sets the Default field in the VersionDefinition to the specified
@@ -13261,35 +13254,6 @@
TYPES
-type DataBytesModifier struct {
- // Has unexported fields.
-}
-
-func NewDataBytesModifier(data []byte) *DataBytesModifier
-
-func (d *DataBytesModifier) Data(data []byte)
-
-func (d *DataBytesModifier) Reset()
-
-func (d *DataBytesModifier) RestoreUnicodeEscapesFromRE2()
- RestoreUnicodeEscapesFromRE2 translates RE2-compatible hexadecimal escape
- sequences (`\x{XXXX}`) back to their original ECMA-262 compliant Unicode
- escape sequence representation (`\uXXXX`). This function is typically used
- when exporting an API definition or any other data structure where regex
- patterns were previously sanitized for internal use with Go's RE2 engine.
- It ensures that external consumers of the data receive the regex patterns in
- their original, more widely supported format.
-
-func (d *DataBytesModifier) Result() []byte
-
-func (d *DataBytesModifier) TransformUnicodeEscapesToRE2()
- TransformUnicodeEscapesToRE2 transforms ECMA-262 compliant Unicode escape
- sequences (`\uXXXX`) into a format that is compatible with Go's RE2 regex
- engine (`\x{XXXX}`). This is necessary because RE2 does not support the `\u`
- escape sequence but does support hexadecimal escapes, which can represent
- any Unicode code point. The function returns a new byte array with the
- transformed pattern.
-
type VersionParameter int
VersionParameter represents the type of parameter used in API version
configuration. It defines the possible parameters that can be used when
@@ -13525,6 +13489,36 @@
func (c CustomPolicyId) Validate() error
+# Package: ./pkg/schema
+
+package schema // import "github.com/TykTechnologies/tyk/pkg/schema"
+
+
+FUNCTIONS
+
+func RestoreUnicodeEscapesFromRE2(str string) string
+func RestoreUnicodeEscapesFromRE2Manipulation(schema *openapi3.Schema)
+func RestoreUnicodeEscapesInError(err error) error
+func TransformUnicodeEscapesToRE2Manipulation(schema *openapi3.Schema)
+
+TYPES
+
+type Manipulation func(schema *openapi3.Schema)
+
+type Manipulations []Manipulation
+
+type Visitor struct {
+ // Has unexported fields.
+}
+
+func NewVisitor() *Visitor
+
+func (v *Visitor) AddSchemaManipulation(manipulation Manipulation)
+
+func (v *Visitor) ProcessOAS(doc *oas.OAS)
+
+func (v *Visitor) ProcessSchema(schemaRef *openapi3.SchemaRef)
+
# Package: ./pkg/validator
package validator // import "github.com/TykTechnologies/tyk/pkg/validator" |
…on-oas-api-v2' into TT-12238-javascript-regex-issue-on-oas-api-v2
|
/Visor ask Changes made by Visitor to OAS struct are then assigned to |
pvormste
left a comment
pvormste
left a comment
There was a problem hiding this comment.
Love this solution, fantastic!
🚨 Jira Linter FailedCommit: The Jira linter failed to validate your PR. Please check the error details below: 🔍 Click to view error detailsNext Steps
This comment will be automatically deleted once the linter passes. |
|
|
/release to release-5.8 |
|
|
pkg/schema was introduced on master by PR #7904 and is not available on release branches. Replace schema.RestoreUnicodeEscapesInError(err) with plain err to match existing release branch error handling. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
pkg/schema was introduced on master by PR #7904 and is not available on release branches. Replace schema.RestoreUnicodeEscapesInError(err) with plain err to match existing release branch error handling. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
4 participants
Description
Related Issue
Motivation and Context
How This Has Been Tested
Screenshots (if appropriate)
Types of changes
Checklist
Ticket Details
TT-12238
Generated at: 2026-03-20 14:48:59