Skip to content

Commit f8b714b

Browse files
authored
fix: enforce propertyNames on extra-allow generated models (#66)
signals.json declares propertyNames (reverse-domain keys) alongside named properties and additionalProperties:true. datamodel-code-generator emits this as class Signals(BaseModel) with model_config=ConfigDict(extra="allow") and the two named fields, so unknown (extra) keys are never checked against the key pattern. Observed: Signals(**{"dev.ucp.buyer_ip": "1.2.3.4", "bogus KEY!": "x"}) is accepted and "bogus KEY!" is kept in model_extra. Expected: the malformed key is rejected, because signals.json requires every property name to match the reverse-domain pattern ^[a-z][a-z0-9]*(?:\.[a-z][a-z0-9_]*)+$ (signals.json propertyNames.pattern, the same pattern as reverse_domain_name.json). Well-formed reverse-domain extras (e.g. com.example.device_id) must still be preserved under extra="allow". The already-enforced sibling case is the dict-keyed maps (e.g. ucp.json capabilities/services/payment_handlers/supported_versions), emitted as dict[ReverseDomainName, V] where pydantic validates the keys. The gap is only the extra-allow BaseModel shape: propertyNames declared on an object that also has named properties. This extends the post-generation model_validator approach added for minProperties (#49-class): postprocess_models.py scans the preprocessed schemas for objects that declare propertyNames AND carry named properties, reads the key pattern from the source schema (inline pattern or a $ref to e.g. reverse_domain_name.json, never duplicated in code), and injects a model_validator(mode="after") that matches every model_extra key against the pattern with re.fullmatch. fullmatch (not re.match) is used so a $-anchored pattern does not admit a trailing newline (re.match lets $ match before a final \n); this agrees with pydantic-core / ECMA-262 (JSON Schema's regex dialect) key semantics, the same behavior the dict-keyed map path already applies. The check reaches the base model and its generated request variants (Signals, SignalsCreateRequest, SignalsUpdateRequest, SignalsCompleteRequest). Out of scope: identity_linking.json scopes map degrades to Any due to an allOf/$ref resolution problem, so no propertyNames-bearing model is emitted for it (a separate defect); and the dict-keyed maps above already enforce their key pattern. Regenerated with ./generate_models.sh 2026-04-08; the diff is limited to the propertyNames enforcement on the four Signals models and regeneration is byte-identical.
1 parent c4d6bcf commit f8b714b

6 files changed

Lines changed: 454 additions & 7 deletions

File tree

postprocess_models.py

Lines changed: 216 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414

1515
"""Post-generation fixes for constraints datamodel-code-generator ignores.
1616
17-
Three constraint families are handled:
17+
Four constraint families are handled:
1818
1919
* ``minProperties`` on an object schema WITH declared properties is dropped by
2020
the generator (issue #49): every field is optional, so an empty instance
@@ -49,6 +49,18 @@
4949
applied to the base model and to its generated request variants (linked by file
5050
stem), and travels wherever the alias is reused as a field type.
5151
52+
* ``propertyNames`` on an object WITH named ``properties`` is not enforced. Such
53+
a schema is emitted as a ``BaseModel(extra="allow")`` with the named fields, so
54+
unknown (extra) keys are accepted without being checked against the declared
55+
key pattern (``signals.json`` requires reverse-domain keys, yet a malformed
56+
extra key validates). The script scans for objects that declare
57+
``propertyNames`` AND carry named ``properties`` and injects a
58+
``model_validator(mode="after")`` that matches every ``model_extra`` key against
59+
the pattern. The pattern is read from the source schema (inline or via ``$ref``
60+
to e.g. ``reverse_domain_name.json``), never duplicated here. An object with
61+
``propertyNames`` but *no* named properties is emitted as a ``dict[KeyType, V]``
62+
whose key type already carries the pattern, so it is out of scope.
63+
5264
* ``uniqueItems`` on an array is dropped entirely by the generator, so a list
5365
field accepts duplicate entries in violation of the schema. The script
5466
collects the names of array properties declared with ``uniqueItems`` and
@@ -86,6 +98,23 @@ def {marker}(self):
8698
return self
8799
'''
88100

101+
_PROPNAMES_MARKER = "_enforce_property_names"
102+
103+
_PROPNAMES_VALIDATOR_TEMPLATE = '''
104+
@model_validator(mode="after")
105+
def {marker}(self):
106+
"""JSON Schema propertyNames: every extra key must match the
107+
declared reverse-domain pattern (schema propertyNames)."""
108+
pattern = {pattern!r}
109+
for key in self.model_extra or {{}}:
110+
if re.fullmatch(pattern, key) is None:
111+
raise ValueError(
112+
f"Property name {{key!r}} does not match the schema "
113+
f"propertyNames pattern {{pattern}}"
114+
)
115+
return self
116+
'''
117+
89118
_UNIQUE_MARKER = "_enforce_unique_items"
90119

91120
_UNIQUE_VALIDATOR_TEMPLATE = '''
@@ -144,6 +173,153 @@ def _ensure_pydantic_import(source, symbol):
144173
)
145174

146175

176+
def _ensure_stdlib_import(source, statement):
177+
"""Add a top-level ``import`` statement if absent.
178+
179+
Inserted right after ``from __future__ import annotations`` so ruff's
180+
isort pass (run later in the pipeline) settles it into the stdlib group.
181+
"""
182+
if re.search(rf"^{re.escape(statement)}$", source, re.M):
183+
return source
184+
return re.sub(
185+
r"^(from __future__ import annotations\n)",
186+
lambda m: f"{m.group(1)}\n{statement}\n",
187+
source,
188+
count=1,
189+
flags=re.M,
190+
)
191+
192+
193+
def _resolve_property_names_pattern(prop_names, schema_path):
194+
"""Return the key pattern a ``propertyNames`` node enforces, or ``None``.
195+
196+
Reads an inline ``pattern`` directly, or follows a ``$ref`` to an external
197+
schema file's root ``pattern`` (e.g. ``reverse_domain_name.json``) so the
198+
pattern is never duplicated here — it always comes from the source schema.
199+
Local ``#/...`` pointer refs are not resolved and are skipped with a
200+
warning rather than guessed.
201+
"""
202+
if not isinstance(prop_names, dict):
203+
return None
204+
inline = prop_names.get("pattern")
205+
if isinstance(inline, str):
206+
return inline
207+
ref = prop_names.get("$ref")
208+
if not isinstance(ref, str):
209+
return None
210+
if ref.startswith("#"):
211+
sys.stderr.write(
212+
f" ! {schema_path}: propertyNames $ref '{ref}' is a local "
213+
"pointer; pattern not resolved\n"
214+
)
215+
return None
216+
file_part = ref.split("#", 1)[0]
217+
target = (Path(schema_path).parent / file_part).resolve()
218+
try:
219+
referenced = json.loads(target.read_text(encoding="utf-8"))
220+
except (OSError, json.JSONDecodeError):
221+
sys.stderr.write(
222+
f" ! {schema_path}: propertyNames $ref '{ref}' could not be "
223+
"loaded; pattern not resolved\n"
224+
)
225+
return None
226+
pattern = (
227+
referenced.get("pattern") if isinstance(referenced, dict) else None
228+
)
229+
if not isinstance(pattern, str):
230+
sys.stderr.write(
231+
f" ! {schema_path}: propertyNames $ref '{ref}' target has no "
232+
"root pattern; not resolved\n"
233+
)
234+
return None
235+
return pattern
236+
237+
238+
def find_property_names_patterns(schema_dir):
239+
"""Map generated class name -> propertyNames pattern for extra-allow models.
240+
241+
The gap this targets: an object schema that declares ``propertyNames`` AND
242+
carries named ``properties`` is emitted by the generator as a
243+
``BaseModel(extra="allow")`` with those named fields, so unknown (extra)
244+
keys are never pattern-checked. An object with ``propertyNames`` but *no*
245+
named ``properties`` is emitted as a ``dict[KeyType, V]`` map whose key type
246+
already carries the pattern (pydantic validates the keys), so it is out of
247+
scope. The class is defined mechanically: has ``propertyNames`` (resolvable
248+
to a pattern) AND non-empty ``properties`` AND a ``title`` to map to a class.
249+
Nested titled objects are walked too, so the rule is general, not per-file.
250+
"""
251+
found = {}
252+
253+
def walk(node, path_str):
254+
if not isinstance(node, dict):
255+
if isinstance(node, list):
256+
for item in node:
257+
walk(item, path_str)
258+
return
259+
props = node.get("properties")
260+
if "propertyNames" in node and isinstance(props, dict) and props:
261+
pattern = _resolve_property_names_pattern(
262+
node["propertyNames"], path_str
263+
)
264+
title = node.get("title")
265+
if pattern is None:
266+
pass
267+
elif not title:
268+
sys.stderr.write(
269+
f" ! {path_str}: propertyNames on an extra-allow object "
270+
"but no title; cannot map to a class\n"
271+
)
272+
else:
273+
# The injected validator uses re.fullmatch to mirror
274+
# pydantic-core / ECMA-262 (JSON Schema's regex dialect) key
275+
# semantics, which the sibling dict-map path already applies.
276+
# That is exact for the ^...$-anchored patterns UCP uses. An
277+
# unanchored pattern means JSON Schema unanchored-search
278+
# semantics, where fullmatch would over-restrict; warn so a
279+
# future schema does not silently get a stricter check.
280+
if not (pattern.startswith("^") and pattern.endswith("$")):
281+
sys.stderr.write(
282+
f" ! {path_str}: propertyNames pattern {pattern!r} is "
283+
"not ^/$-anchored; fullmatch enforcement may be "
284+
"stricter than JSON Schema search semantics\n"
285+
)
286+
found[_alias_name(title)] = pattern
287+
for value in node.values():
288+
walk(value, path_str)
289+
290+
for path in sorted(Path(schema_dir).rglob("*.json")):
291+
try:
292+
schema = json.loads(path.read_text(encoding="utf-8"))
293+
except (OSError, json.JSONDecodeError):
294+
continue
295+
walk(schema, str(path))
296+
return found
297+
298+
299+
def inject_property_names(source, class_name, pattern):
300+
"""Inject the propertyNames key validator at the end of ``class_name``."""
301+
class_re = re.compile(rf"^class {re.escape(class_name)}\(", re.M)
302+
match = class_re.search(source)
303+
if not match:
304+
return source
305+
# The class body ends at the next top-level statement or EOF.
306+
tail = re.compile(r"^\S", re.M)
307+
end_match = tail.search(source, match.end())
308+
end = end_match.start() if end_match else len(source)
309+
# Scope the idempotency guard to this class's own body, so a second
310+
# target class in the same module is still patched.
311+
if f"def {_PROPNAMES_MARKER}(" in source[match.start() : end]:
312+
return source
313+
method = _PROPNAMES_VALIDATOR_TEMPLATE.format(
314+
marker=_PROPNAMES_MARKER, pattern=pattern
315+
)
316+
body = source[:end].rstrip("\n")
317+
rest = source[end:]
318+
out = body + "\n" + method + ("\n" + rest if rest else "")
319+
out = _ensure_pydantic_import(out, "model_validator")
320+
return _ensure_stdlib_import(out, "import re")
321+
322+
147323
def inject_min_properties(source, class_name, minimum):
148324
"""Inject the minProperties validator at the end of ``class_name``."""
149325
if f"def {_MARKER}(" in source:
@@ -540,6 +716,42 @@ def _array_contains_targets():
540716
return targets
541717

542718

719+
def _patch_property_names():
720+
"""Inject propertyNames validators; return (patched_count, exit_code)."""
721+
patterns = find_property_names_patterns(SCHEMA_DIR)
722+
if not patterns:
723+
sys.stdout.write(
724+
"postprocess: no propertyNames constraints on extra-allow "
725+
"models found\n"
726+
)
727+
return 0, 0
728+
patched = 0
729+
for class_name, pattern in sorted(patterns.items()):
730+
hits = []
731+
for path in sorted(OUTPUT_DIR.rglob("*.py")):
732+
source = path.read_text(encoding="utf-8")
733+
if not re.search(
734+
rf"^class {re.escape(class_name)}\(", source, re.M
735+
):
736+
continue
737+
updated = inject_property_names(source, class_name, pattern)
738+
if updated != source:
739+
path.write_text(updated, encoding="utf-8")
740+
patched += 1
741+
hits.append(path)
742+
label = ", ".join(str(h) for h in hits) or "NO GENERATED CLASS FOUND"
743+
sys.stdout.write(
744+
f" propertyNames {pattern!r} on '{class_name}' -> {label}\n"
745+
)
746+
if not hits:
747+
sys.stderr.write(
748+
f" ! '{class_name}' has no generated class; "
749+
"constraint not enforced\n"
750+
)
751+
return patched, 1
752+
return patched, 0
753+
754+
543755
def _patch_array_contains():
544756
"""Inject array-contains validators; return (patched_count, exit_code)."""
545757
targets = _array_contains_targets()
@@ -606,11 +818,12 @@ def _patch_unique_items():
606818
def main():
607819
"""Main entry point to scan schemas and patch generated models."""
608820
patched_mp, rc_mp = _patch_min_properties()
821+
patched_pn, rc_pn = _patch_property_names()
609822
patched_ac, rc_ac = _patch_array_contains()
610823
patched_ui, rc_ui = _patch_unique_items()
611-
total = patched_mp + patched_ac + patched_ui
824+
total = patched_mp + patched_pn + patched_ac + patched_ui
612825
sys.stdout.write(f"postprocess: {total} module(s) patched\n")
613-
return rc_mp or rc_ac or rc_ui
826+
return rc_mp or rc_pn or rc_ac or rc_ui
614827

615828

616829
if __name__ == "__main__":

src/ucp_sdk/models/schemas/shopping/types/signals.py

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,9 @@
1818

1919
from __future__ import annotations
2020

21-
from pydantic import BaseModel, ConfigDict, Field
21+
import re
22+
23+
from pydantic import BaseModel, ConfigDict, Field, model_validator
2224

2325

2426
class Signals(BaseModel):
@@ -37,3 +39,16 @@ class Signals(BaseModel):
3739
"""
3840
Client's HTTP User-Agent header or equivalent.
3941
"""
42+
43+
@model_validator(mode="after")
44+
def _enforce_property_names(self):
45+
"""JSON Schema propertyNames: every extra key must match the
46+
declared reverse-domain pattern (schema propertyNames)."""
47+
pattern = "^[a-z][a-z0-9]*(?:\\.[a-z][a-z0-9_]*)+$"
48+
for key in self.model_extra or {}:
49+
if re.fullmatch(pattern, key) is None:
50+
raise ValueError(
51+
f"Property name {key!r} does not match the schema "
52+
f"propertyNames pattern {pattern}"
53+
)
54+
return self

src/ucp_sdk/models/schemas/shopping/types/signals_complete_request.py

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,9 @@
1818

1919
from __future__ import annotations
2020

21-
from pydantic import BaseModel, ConfigDict, Field
21+
import re
22+
23+
from pydantic import BaseModel, ConfigDict, Field, model_validator
2224

2325

2426
class SignalsCompleteRequest(BaseModel):
@@ -37,3 +39,16 @@ class SignalsCompleteRequest(BaseModel):
3739
"""
3840
Client's HTTP User-Agent header or equivalent.
3941
"""
42+
43+
@model_validator(mode="after")
44+
def _enforce_property_names(self):
45+
"""JSON Schema propertyNames: every extra key must match the
46+
declared reverse-domain pattern (schema propertyNames)."""
47+
pattern = "^[a-z][a-z0-9]*(?:\\.[a-z][a-z0-9_]*)+$"
48+
for key in self.model_extra or {}:
49+
if re.fullmatch(pattern, key) is None:
50+
raise ValueError(
51+
f"Property name {key!r} does not match the schema "
52+
f"propertyNames pattern {pattern}"
53+
)
54+
return self

src/ucp_sdk/models/schemas/shopping/types/signals_create_request.py

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,9 @@
1818

1919
from __future__ import annotations
2020

21-
from pydantic import BaseModel, ConfigDict, Field
21+
import re
22+
23+
from pydantic import BaseModel, ConfigDict, Field, model_validator
2224

2325

2426
class SignalsCreateRequest(BaseModel):
@@ -37,3 +39,16 @@ class SignalsCreateRequest(BaseModel):
3739
"""
3840
Client's HTTP User-Agent header or equivalent.
3941
"""
42+
43+
@model_validator(mode="after")
44+
def _enforce_property_names(self):
45+
"""JSON Schema propertyNames: every extra key must match the
46+
declared reverse-domain pattern (schema propertyNames)."""
47+
pattern = "^[a-z][a-z0-9]*(?:\\.[a-z][a-z0-9_]*)+$"
48+
for key in self.model_extra or {}:
49+
if re.fullmatch(pattern, key) is None:
50+
raise ValueError(
51+
f"Property name {key!r} does not match the schema "
52+
f"propertyNames pattern {pattern}"
53+
)
54+
return self

src/ucp_sdk/models/schemas/shopping/types/signals_update_request.py

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,9 @@
1818

1919
from __future__ import annotations
2020

21-
from pydantic import BaseModel, ConfigDict, Field
21+
import re
22+
23+
from pydantic import BaseModel, ConfigDict, Field, model_validator
2224

2325

2426
class SignalsUpdateRequest(BaseModel):
@@ -37,3 +39,16 @@ class SignalsUpdateRequest(BaseModel):
3739
"""
3840
Client's HTTP User-Agent header or equivalent.
3941
"""
42+
43+
@model_validator(mode="after")
44+
def _enforce_property_names(self):
45+
"""JSON Schema propertyNames: every extra key must match the
46+
declared reverse-domain pattern (schema propertyNames)."""
47+
pattern = "^[a-z][a-z0-9]*(?:\\.[a-z][a-z0-9_]*)+$"
48+
for key in self.model_extra or {}:
49+
if re.fullmatch(pattern, key) is None:
50+
raise ValueError(
51+
f"Property name {key!r} does not match the schema "
52+
f"propertyNames pattern {pattern}"
53+
)
54+
return self

0 commit comments

Comments
 (0)