Skip to content

Compute Context.module_names lazily to avoid rescan on every evolve() - #4586

Merged
kddejong merged 3 commits into
aws-cloudformation:mainfrom
sir-sigurd:perf-lazy-module-names
Jul 23, 2026
Merged

Compute Context.module_names lazily to avoid rescan on every evolve()#4586
kddejong merged 3 commits into
aws-cloudformation:mainfrom
sir-sigurd:perf-lazy-module-names

Conversation

@sir-sigurd

Copy link
Copy Markdown
Contributor

Issue #, if available: #4585

Description of changes:

Context.__post_init__ rebuilt module_names on every Context construction — including all evolve() clones created during the schema walk — making validation O(nodes × resources) on every template (#4510, first released in 1.51.1). See the linked issue for profiles: on a ~250 KB template with ~270 resources this is ~84k constructions triggering ~22.4M str.endswith calls, ~46% of total runtime.

This change replaces the eager field with functools.cached_property, so the tuple is computed at most once per instance and only when actually read. Both readers (dynamicValidation in _keywords_cfn.py and GetAtt) consult it only at specific nodes, so most instances never compute it at all. cached_property works on the frozen dataclass because it writes to the instance __dict__ directly, bypassing the frozen __setattr__.

Behavior note: module_names is no longer a dataclass field, so it is not part of the generated __eq__/__repr__. It is fully derived from resources (which is a field), and nothing in the codebase introspects it as a field.

Benchmarks (Apple M-series, warm): synthetic 400-resource template 4.8s → 3.7s (pre-regression 1.51.0 baseline: 3.4s); a fleet of 66 large (~250 KB) generated templates 3m55s → 2m42s. Unit test suite and pre-commit hooks pass.


Prepared with AI assistance (Claude Code); human-reviewed before submission.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

sir-sigurd and others added 3 commits July 23, 2026 17:43
Context.__post_init__ rescanned all resources to rebuild module_names
on every Context construction. Context.evolve() constructs a new
Context on nearly every schema-walk descend, so validation cost grew
with nodes x resources (introduced in aws-cloudformation#4510, first released in
v1.51.1). On a 250KB template with ~270 resources this is ~84k
constructions triggering ~22M str.endswith calls, ~46% of total
runtime.

Replace the eager field with functools.cached_property so the tuple is
computed at most once per instance, and only when actually read (the
two readers only consult it at specific nodes, so most instances never
compute it at all).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Asserts that Context construction and evolve() do not scan resources,
and that module_names is computed on first read and cached per
instance. Fails against the eager __post_init__ implementation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.42%. Comparing base (9c2445f) to head (7ed69ee).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4586      +/-   ##
==========================================
- Coverage   94.43%   94.42%   -0.01%     
==========================================
  Files         429      429              
  Lines       15099    15099              
  Branches     2916     2916              
==========================================
- Hits        14258    14257       -1     
  Misses        461      461              
- Partials      380      381       +1     
Flag Coverage Δ
unittests 94.42% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kddejong
kddejong merged commit 8829729 into aws-cloudformation:main Jul 23, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants