OAS 3.1 import rejects default FastAPI/Pydantic output: lookahead `pattern` on Decimal fields fails RE2 compile (invalid or unsupported Perl syntax: (?!)

[sakaal]

Summary

Importing an OpenAPI 3.1.0 document produced by the default FastAPI + Pydantic v2
toolchain fails component validation when a schema pattern uses a lookahead. The
pattern is valid ECMA-262 (the dialect OAS 3.1 / JSON Schema 2020-12 defines pattern
against), but Tyk's vendored kin-openapi compiles patterns with Go's regexp (RE2),
which has no lookarounds. FastAPI/Pydantic generates these patterns automatically for
ordinary Decimal fields — users do not write them — so this affects a large share of
real-world Python services out of the box.

Environment

• Tyk Gateway 5.13.0
• Spec generated by FastAPI 0.137.1 / Pydantic 2.13.4 (pydantic-core 2.46.4), Python 3.11
• OpenAPI 3.1.0 (FastAPI default since 0.99; with Pydantic v2 since 0.100)
• Reproduces on Pydantic >= 2.12.0 (pydantic-core >= 2.41), which is when Pydantic began
  emitting this pattern. Pydantic <= 2.11.x does not emit it.

Error

invalid components: schema "Package": error parsing regexp: invalid or unsupported Perl syntax: `(?!`

Observed when importing a FastAPI-generated OpenAPI 3.1.0 document into Tyk Gateway 5.13.0.

Minimal repro

A nullable Decimal field, exactly as Pydantic 2.12+ emits it:

{
  "value": {
    "anyOf": [
      { "type": "string", "pattern": "^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$" },
      { "type": "null" }
    ]
  }
}

A Decimal(max_digits=10, decimal_places=2) field emits a second lookahead form:

^(?!^[-+.]*$)[+-]?0*(?:\d{0,8}|(?=[\d.]{1,11}0*$)\d{0,8}\.\d{0,2}0*$)

Both compile in V8 (new RegExp(...)) but not in Go's regexp (RE2 has no lookaround):
regexp.Compile reports error parsing regexp: invalid or unsupported Perl syntax: `(?!` .

Root cause

OpenAPI 3.1.0 adopts JSON Schema 2020-12, whose pattern keyword is defined against the
ECMA-262 regex dialect; lookaheads are valid ECMA-262. Tyk validates via the vendored
kin-openapi fork, which compiles every pattern with Go's standard-library regexp
(RE2). RE2 is a strict subset of ECMA-262 — it deliberately omits lookarounds,
lookbehinds and backreferences (to guarantee linear-time matching) and caps bounded
repetition at 1000. So a fully ECMA-262-valid pattern cannot compile under RE2, and the
whole document is rejected at import-time document/component validation (doc.Validate()),
distinct from request-time validation.

Why this matters

The constraint is framework-generated, so the JSON Schema guidance that authors keep to a
portable regex subset has no one to act on — there is no author in the loop, and users
cannot reasonably be expected to hand-edit Pydantic's generated schemas. A single
uncompilable pattern currently fails the entire import.

Requested behaviour

1. Don't crash: the same construct triggers a nil-pointer panic in request validation
   upstream (Invalid regex pattern fails silently on openapi3filter ValidateResponse/ValidateRequest then panics getkin/kin-openapi#1044).
2. Don't reject the whole document over one uncompilable pattern — skip that single
   constraint with a warning (graceful degradation, consistent with Tighten spec for regex patterns, to improve interoperability json-schema-org/json-schema-spec#816),
   so default FastAPI/Pydantic specs import cleanly.
3. Optionally, offer an opt-in richer regex engine (e.g. regexp2) for users who need
   lookaround.

Related

• Invalid regex pattern fails silently on openapi3filter ValidateResponse/ValidateRequest then panics getkin/kin-openapi#1044 — same root cause and identical error string; reported as a
  nil-pointer panic in openapi3filter.ValidateRequest at request-validation time.
• Error in pattern validation  getkin/kin-openapi#409 — same import-time failure mode, different trigger (Go's
  repeat-count cap on {0,4096}), showing this is a recurring RE2-subset class.
• Decimal JSON schema pattern uses lookahead regex unsupported by pydantic-core pydantic/pydantic#13017 — the upstream source of the lookahead pattern; closed but
  acknowledges RE2-class engines cannot compile it.
• Decimal JSON schema pattern rejects pydantic's own scientific notation serialization output (v2.12 regression) pydantic/pydantic#13089 — the same generated pattern is independently buggy (rejects
  scientific notation Pydantic itself serializes).

A conformant, RE2-safe reference

The problem is fully avoidable on the spec-producing side: rewrite the auto-generated
Decimal pattern to an equivalent that stays inside the RE2 subset, leaving runtime
validation untouched (the emitted pattern is only a JSON-Schema annotation; Pydantic's
input validation does not use it). Done via Pydantic's public __get_pydantic_json_schema__
hook with no change to regex_engine.

The plain-Decimal case becomes:

^[+-]?(?:\d+\.?\d*|\.\d+)$

Constrained cases (max_digits / decimal_places) encode the digit budget by enumerating
the integer/fraction split instead of using a lookahead, e.g. max_digits=10, decimal_places=2 becomes:

^[+-]?(?:0*\d{1,8}(?:\.\d{0,2}0*)?|0*\.\d{1,2}0*)$

Every such pattern was verified to compile under both Go's RE2 and V8 (ECMA-262), and to
accept/reject the same decimal strings as Pydantic's original. Sharing this only as
evidence that conformant-and-RE2-safe output exists — the fix that unblocks the broad
population of FastAPI/Pydantic users belongs in Tyk's import path (graceful degradation),
not in every user's schema.

[probelabs]

Hello @sakaal,

Thank you for this exceptionally detailed and well-researched bug report. Your analysis of the root cause is excellent and provides a very clear picture of the issue.

Summary of Understanding

Based on your report, we understand that:

• Tyk's OpenAPI (OAS) 3.1 importer fails when it encounters a spec generated by the default FastAPI/Pydantic toolchain.
• The failure is due to a regex pattern containing a negative lookahead ((?!...), which is valid under the ECMA-262 dialect required by the OAS 3.1 specification.
• Tyk's validation, which relies on Go's standard regexp package (RE2), does not support lookarounds, causing a compilation error that rejects the entire API definition.
• This blocks a significant and common use case for modern Python-based services out of the box.

Next Steps

Your diagnosis appears to be correct. The core of the problem lies in the incompatibility between the regex engine used for validation and the patterns allowed by the specification. We agree that the ideal behavior would be to handle this more gracefully.

Your suggestion to degrade gracefully by skipping the single uncompilable pattern constraint with a warning is a very sensible approach. This would unblock the import of these specifications while still informing the user that a specific validation rule could not be enforced.

The team will need to investigate the feasibility of implementing this within our OAS validation pipeline. This will likely involve looking at how we use our kin-openapi dependency and where we can intercept these compilation errors.

Thank you again for the high-quality report and the suggested workarounds. It is incredibly helpful.

References

• tyk/apidef/validator.go:121-133 – Location where the OpenAPI document validation is initiated.

---

Powered by Visor from Probelabs

💡 TIP: You can chat with Visor using /visor ask <your question>