feat: Optional caching for read_terragrunt_config to improve performance #5116
Conversation
jkarkoszka
requested review from
ThisGuyCodes,
denis256 and
yhakbar
as code owners
|
@jkarkoszka-wipro is attempting to deploy a commit to the Gruntwork Team on Vercel. A member of the Team first needs to authorize it. |
📝 WalkthroughWalkthroughAdds a cached variant read_terragrunt_config_with_cache, a per-context cache key and initialization, wires the cached reader into the eval context, adds documentation, multiple test fixtures exercising cache/defaults/dependencies/original-dir behavior, and new integration tests validating cached reads. Changes
Sequence Diagram(s)sequenceDiagram
participant HCL as HCL Eval
participant Func as read_terragrunt_config_with_cache
participant Cache as Per-Context Cache
participant Parser as ParseTerragruntConfig
rect rgb(245, 250, 255)
Note over HCL,Parser: First call (cache miss)
HCL->>Func: read_terragrunt_config_with_cache(path [, default])
Func->>Cache: lookup(key = path + hasDefault)
Cache-->>Func: miss
Func->>Parser: ParseTerragruntConfig(ctx, logger, path, default)
Parser-->>Func: cty.Value
Func->>Cache: store(key, cty.Value)
Func-->>HCL: return cty.Value
end
rect rgb(245, 255, 245)
Note over HCL,Cache: Subsequent call (cache hit)
HCL->>Func: read_terragrunt_config_with_cache(path [, default])
Func->>Cache: lookup(key)
Cache-->>Func: hit (cty.Value)
Func-->>HCL: return cached cty.Value
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Possibly related issues
Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
docs-starlight/src/content/docs/04-reference/01-hcl/04-functions.mdx (2)
907-944: Clarify cache scope and invalidation.State that the cache is per Terragrunt invocation (in‑memory) and not persisted across processes, and that first-parse values won’t reflect file edits later in the same invocation. This avoids confusion about lifetime and “stale” reads.
Apply this doc tweak:
## read_terragrunt_config_with_cache @@ -`read_terragrunt_config_with_cache(config_path, [default_val])` is similar to `read_terragrunt_config`, but it caches the parsed configuration to avoid re-parsing the same file multiple times during configuration parsing. This can improve performance when the same configuration file is read multiple times. +`read_terragrunt_config_with_cache(config_path, [default_val])` is similar to `read_terragrunt_config`, but it caches the parsed configuration to avoid re-parsing the same file multiple times during a single Terragrunt invocation. The cache is in‑memory (per process) and is not persisted between runs. This can improve performance when the same configuration file is read multiple times. @@ -**Note:** The cached values are preserved from the first time the configuration is parsed, which means that if the same file is read multiple times, subsequent reads will return the cached value from the first read, not a fresh parse. This is expected behavior and proves that the cache is working correctly. +**Scope & invalidation:** The first successful parse “wins.” Subsequent reads in the same invocation return that cached value, even if the underlying file changes later during parsing. A new Terragrunt process starts with an empty cache.
939-949: Document parity with stack/values files.Mirror the note from
read_terragrunt_configthat this cached variant can also readterragrunt.stack.hclandterragrunt.values.hcl, since the implementation routes through the same parser.Suggested addition:
@@ If you need the `original_terragrunt_dir` to reflect the current parsing context, use `read_terragrunt_config` instead of `read_terragrunt_config_with_cache`. +Notes: + +- `read_terragrunt_config_with_cache` can also be used to read `terragrunt.stack.hcl` and `terragrunt.values.hcl` files, with the same behavior as `read_terragrunt_config`.test/integration_test.go (1)
2798-2930: Optional: factor common assertions to a helper.There’s duplicated JSON unmarshal + large structure assertions across full/non-cached versions. A small helper (e.g.,
assertFullOutputs(t, outputs)) would reduce drift.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
docs-starlight/public/d2/docs/03-features/05-run-queue-0.svgis excluded by!**/*.svg
📒 Files selected for processing (23)
config/config_helpers.go(3 hunks)config/context.go(1 hunks)docs-starlight/src/content/docs/04-reference/01-hcl/04-functions.mdx(1 hunks)docs-starlight/src/data/commands/find.mdx(1 hunks)docs-starlight/src/data/flags/find-reading.mdx(1 hunks)test/fixtures/read-config-with-cache/from_dependency/dep/main.tf(1 hunks)test/fixtures/read-config-with-cache/from_dependency/dep/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/from_dependency/dep/vars.hcl(1 hunks)test/fixtures/read-config-with-cache/from_dependency/main.tf(1 hunks)test/fixtures/read-config-with-cache/from_dependency/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/full/main.tf(1 hunks)test/fixtures/read-config-with-cache/full/source.hcl(1 hunks)test/fixtures/read-config-with-cache/full/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_default/main.tf(1 hunks)test/fixtures/read-config-with-cache/with_default/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_dependency/dep/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_dependency/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/main.tf(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/foo/bar.hcl(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/main.tf(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcl(1 hunks)test/integration_test.go(2 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
docs-starlight/**/*.md*
⚙️ CodeRabbit configuration file
Review the documentation for clarity, grammar, and spelling. Make sure that the documentation is easy to understand and follow. There is currently a migration underway from the Jekyll based documentation in
docsto the Starlight + Astro based documentation indocs-starlight. Make sure that thedocs-starlightdocumentation is accurate and up-to-date with thedocsdocumentation, and that any difference between them results in an improvement in thedocs-starlightdocumentation.
Files:
docs-starlight/src/data/commands/find.mdxdocs-starlight/src/data/flags/find-reading.mdxdocs-starlight/src/content/docs/04-reference/01-hcl/04-functions.mdx
**/*.go
⚙️ CodeRabbit configuration file
Review the Go code for quality and correctness. Make sure that the Go code follows best practices, is performant, and is easy to understand and maintain.
Files:
config/config_helpers.gotest/integration_test.goconfig/context.go
🧠 Learnings (2)
📚 Learning: 2025-02-10T23:20:04.295Z
Learnt from: yhakbar
Repo: gruntwork-io/terragrunt PR: 3868
File: docs-starlight/patches/@astrojs%[email protected]:33-33
Timestamp: 2025-02-10T23:20:04.295Z
Learning: In Terragrunt projects, all `.hcl` files can be assumed to be Terragrunt configurations by default, with specific exceptions like `.terraform.lock.hcl` that need explicit handling.
Applied to files:
docs-starlight/src/data/commands/find.mdxtest/fixtures/read-config-with-cache/with_default/terragrunt.hcltest/fixtures/read-config-with-cache/with_dependency/terragrunt.hcltest/fixtures/read-config-with-cache/full/terragrunt.hcltest/fixtures/read-config-with-cache/with_original_terragrunt_dir/foo/bar.hcltest/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcltest/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/terragrunt.hcltest/fixtures/read-config-with-cache/full/source.hcl
📚 Learning: 2025-11-03T04:40:01.000Z
Learnt from: ThisGuyCodes
Repo: gruntwork-io/terragrunt PR: 5041
File: test/fixtures/hclvalidate/valid/duplicate-attributes-in-required-providers/main.tf:2-7
Timestamp: 2025-11-03T04:40:01.000Z
Learning: In the terragrunt repository, test fixtures under test/fixtures/hclvalidate/valid/ are intentionally testing that INPUT validation succeeds even when Terraform code contains syntax errors or other issues unrelated to input validation (e.g., duplicate attributes, circular references, invalid locals). The "valid" designation means "valid for input validation purposes" not "syntactically valid Terraform/OpenTofu code".
Applied to files:
test/fixtures/read-config-with-cache/with_default/terragrunt.hcltest/fixtures/read-config-with-cache/with_dependency/terragrunt.hcltest/fixtures/read-config-with-cache/full/terragrunt.hcltest/fixtures/read-config-with-cache/with_original_terragrunt_dir/foo/bar.hcltest/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcltest/fixtures/read-config-with-cache/with_original_terragrunt_dir/main.tftest/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/terragrunt.hcltest/integration_test.gotest/fixtures/read-config-with-cache/from_dependency/dep/terragrunt.hcl
🧬 Code graph analysis (3)
config/config_helpers.go (5)
config/parsing_context.go (1)
ParsingContext(17-60)config/errors.go (1)
WrongNumberOfParamsError(89-93)options/options.go (1)
TerragruntOptions(99-329)internal/cache/cache.go (1)
ContextCache(121-128)config/context.go (1)
ReadTerragruntConfigCacheContextKey(17-17)
test/integration_test.go (2)
test/helpers/package.go (6)
CleanupTerraformFolder(882-889)CopyEnvironment(89-105)RunTerragrunt(979-983)RunTerragruntCommand(965-969)LogBufferContentsLineByLine(985-993)TerraformOutput(83-87)util/file.go (1)
JoinPath(626-628)
config/context.go (1)
internal/cache/cache.go (1)
NewCache(24-30)
🔇 Additional comments (19)
test/fixtures/read-config-with-cache/from_dependency/terragrunt.hcl (1)
1-7: Test fixture structure is correct.Dependency and inputs mapping follow correct Terragrunt HCL syntax.
docs-starlight/src/data/flags/find-reading.mdx (1)
26-31: Documentation update is consistent and clear.Line 28 adds the cached variant in the same list format as other helper functions. Description accurately conveys the caching purpose.
test/fixtures/read-config-with-cache/from_dependency/dep/main.tf (1)
1-3: Minimal test fixture is appropriate.Simple variable and output structure is suitable for testing the caching mechanism.
test/fixtures/read-config-with-cache/from_dependency/main.tf (1)
1-2: Test fixture follows expected dependency pattern.Variable and output structure mirrors the dependency's foo/bar relationship defined in terragrunt.hcl.
test/fixtures/read-config-with-cache/from_dependency/dep/terragrunt.hcl (1)
1-7: Fixture is complete; vars.hcl exists at the expected location.The
vars.hclfile is present attest/fixtures/read-config-with-cache/from_dependency/dep/vars.hcl, confirming the fixture is properly structured and the reference on line 2 is valid.Likely an incorrect or invalid review comment.
docs-starlight/src/data/commands/find.mdx (1)
257-263: Link target exists and documentation is complete.The function
read_terragrunt_config_with_cacheis properly documented indocs-starlight/src/content/docs/04-reference/01-hcl/04-functions.mdxat line 907 with a matching anchor. The link/docs/reference/hcl/functions/#read_terragrunt_config_with_cachewill correctly resolve. The description aligns with the source documentation.Note: This function is not yet documented in the legacy
docs/reference/hcl/functions.mdfile. If consistency between both documentation systems is required during the migration, the old documentation may need to be updated separately.test/fixtures/read-config-with-cache/full/terragrunt.hcl (1)
1-19: The source.hcl file exists and contains all 13 properties referenced by the terragrunt.hcl fixture. The fixture is complete and properly structured for testing the read_terragrunt_config_with_cache function.Likely an incorrect or invalid review comment.
test/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcl (1)
1-18: Fixture files are present and properly integrated.All referenced files exist and are correctly configured:
foo/bar.hclexportsterragrunt_dirandoriginal_terragrunt_dirdep/terragrunt.hclreads the cached config and exports outputsmain.tfin dep/ provides the outputsThe fixture is integrated into the test suite (test/integration_test.go:2713) with comprehensive test coverage including assertions on terragrunt_dir and original_terragrunt_dir propagation through caching.
Likely an incorrect or invalid review comment.
test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/terragrunt.hcl (1)
1-11: LGTM for the fixture.Matches the cache semantics you’re testing; paths and locals resolve correctly.
config/context.go (1)
13-28: LGTM: context cache wiring looks correct.New key and cache initialized with the right value type (
cty.Value), and added alongside existing caches.Also applies to: 30-36
test/fixtures/read-config-with-cache/full/source.hcl (1)
1-76: LGTM for the fixture.Accurately mirrors the non-cached “full” config for parity tests.
test/integration_test.go (5)
98-99: LGTM: fixture constant addition.Name and path align with the new fixtures.
2603-2652: LGTM: cached read with dependency.Assertions mirror the existing non-cached test and add a safe type check for numbers.
2654-2684: LGTM: cached read from dependency.Matches the non-cached counterpart and validates expected output.
2686-2706: LGTM: cached read with default.Covers the default-path branch for parity with the original function.
2708-2796: Verify determinism of “first-parse wins” in run --all.The last block asserts that, during
run --all apply, the cache preservesoriginal_terragrunt_dirfrom the first parse. Ensure the parse order is deterministic (dependency first or documented) so the expectation (rootPathAbsvsdepPathAbs) can’t flicker with implementation changes.Consider adding a brief comment in the test about the expected evaluation order, or assert against whichever unit parses first (and document why).
config/config_helpers.go (2)
54-55: LGTM: function name constant.Public surface matches docs (
read_terragrunt_config_with_cache).
152-161: LGTM: function registered in eval context.Exposed alongside the original variant; no collisions.
test/fixtures/read-config-with-cache/with_original_terragrunt_dir/main.tf (1)
1-47: LGTM for the fixture.Straightforward vars/outputs mapping used by the integration test.
jkarkoszka
force-pushed
the
feat/cache-for-read-terragrunt-config
branch
from
dc159da to
81f3389
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
docs-starlight/src/content/docs/04-reference/01-hcl/04-functions.mdx (1)
943-943: Minor wording refinement in the caching behavior warning.The content is accurate and important, but the phrasing could be slightly stronger. Consider revising the warning section:
- Line 943: "proves that the cache is working correctly" is weaker than needed. Suggest: "is the intended caching behavior"
- Lines 945-950: The warning is clear, but it could more explicitly state the risk: explicitly note that cached
original_terragrunt_dirvalues can be stale across different calling contextsFor example:
-**Note:** The cached values are preserved from the first time the configuration is parsed, which means that if the same file is read multiple times, subsequent reads will return the cached value from the first read, not a fresh parse. This is expected behavior and proves that the cache is working correctly. +**Note:** The cached values are preserved from the first time the configuration is parsed, which means that if the same file is read multiple times, subsequent reads will return the cached value from the first read, not a fresh parse. This is the intended caching behavior. -**Warning:** When using `read_terragrunt_config_with_cache`, be aware that functions that depend on the original terragrunt directory context (such as [`get_original_terragrunt_dir()`](#get_original_terragrunt_dir)) will return values from the first time the configuration was parsed, not from the current parsing context. +**Warning:** When using `read_terragrunt_config_with_cache`, be aware that context-dependent functions like [`get_original_terragrunt_dir()`](#get_original_terragrunt_dir) will return the stale value from the first parse, not the current parsing context. This can cause issues when the same configuration file is accessed from different modules.Also applies to: 945-950
test/integration_test.go (1)
98-98: New read-config-with-cache tests are consistent with existing coverageThe new
testFixtureReadConfigWithCacheconstant and theTestReadTerragruntConfigWithCache*tests correctly mirror the existing read-config integration tests, just pointed at the cached fixtures. The extra type check foroutputs["number"]inTestReadTerragruntConfigWithCacheWithDependencyis also a nice guard against panics from unexpected JSON output types.One thing to consider (not blocking):
- The WithCache tests currently only assert that
read_terragrunt_config_with_cachereturns the same values as the non-cached variant. If you want to guard against regressions where the function stops caching and simply delegates toread_terragrunt_config, you might later add a more targeted test (unit or integration) that asserts cache reuse (e.g., via logging or a counter) rather than just functional parity.Given the size of this file and the WIP status of the PR, keeping these tests straightforward is reasonable; any deduplication with the non-cache variants (shared helpers parameterized by fixture root) can stay an optional follow-up.
Also applies to: 2603-2930
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (23)
config/config_helpers.go(3 hunks)config/context.go(1 hunks)docs-starlight/src/content/docs/04-reference/01-hcl/04-functions.mdx(1 hunks)docs-starlight/src/data/commands/find.mdx(1 hunks)docs-starlight/src/data/flags/find-reading.mdx(1 hunks)test/fixtures/read-config-with-cache/from_dependency/dep/main.tf(1 hunks)test/fixtures/read-config-with-cache/from_dependency/dep/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/from_dependency/dep/vars.hcl(1 hunks)test/fixtures/read-config-with-cache/from_dependency/main.tf(1 hunks)test/fixtures/read-config-with-cache/from_dependency/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/full/main.tf(1 hunks)test/fixtures/read-config-with-cache/full/source.hcl(1 hunks)test/fixtures/read-config-with-cache/full/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_default/main.tf(1 hunks)test/fixtures/read-config-with-cache/with_default/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_dependency/dep/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_dependency/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/main.tf(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/foo/bar.hcl(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/main.tf(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcl(1 hunks)test/integration_test.go(2 hunks)
✅ Files skipped from review due to trivial changes (1)
- test/fixtures/read-config-with-cache/with_default/terragrunt.hcl
🚧 Files skipped from review as they are similar to previous changes (17)
- test/fixtures/read-config-with-cache/with_dependency/terragrunt.hcl
- test/fixtures/read-config-with-cache/with_original_terragrunt_dir/foo/bar.hcl
- test/fixtures/read-config-with-cache/from_dependency/dep/main.tf
- test/fixtures/read-config-with-cache/full/terragrunt.hcl
- test/fixtures/read-config-with-cache/from_dependency/dep/terragrunt.hcl
- test/fixtures/read-config-with-cache/with_dependency/dep/terragrunt.hcl
- config/context.go
- config/config_helpers.go
- test/fixtures/read-config-with-cache/with_original_terragrunt_dir/main.tf
- test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/main.tf
- docs-starlight/src/data/commands/find.mdx
- test/fixtures/read-config-with-cache/full/source.hcl
- test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/terragrunt.hcl
- test/fixtures/read-config-with-cache/from_dependency/dep/vars.hcl
- test/fixtures/read-config-with-cache/with_default/main.tf
- docs-starlight/src/data/flags/find-reading.mdx
- test/fixtures/read-config-with-cache/full/main.tf
🧰 Additional context used
📓 Path-based instructions (2)
docs-starlight/**/*.md*
⚙️ CodeRabbit configuration file
Review the documentation for clarity, grammar, and spelling. Make sure that the documentation is easy to understand and follow. There is currently a migration underway from the Jekyll based documentation in
docsto the Starlight + Astro based documentation indocs-starlight. Make sure that thedocs-starlightdocumentation is accurate and up-to-date with thedocsdocumentation, and that any difference between them results in an improvement in thedocs-starlightdocumentation.
Files:
docs-starlight/src/content/docs/04-reference/01-hcl/04-functions.mdx
**/*.go
⚙️ CodeRabbit configuration file
Review the Go code for quality and correctness. Make sure that the Go code follows best practices, is performant, and is easy to understand and maintain.
Files:
test/integration_test.go
🧠 Learnings (2)
📚 Learning: 2025-11-03T04:40:01.000Z
Learnt from: ThisGuyCodes
Repo: gruntwork-io/terragrunt PR: 5041
File: test/fixtures/hclvalidate/valid/duplicate-attributes-in-required-providers/main.tf:2-7
Timestamp: 2025-11-03T04:40:01.000Z
Learning: In the terragrunt repository, test fixtures under test/fixtures/hclvalidate/valid/ are intentionally testing that INPUT validation succeeds even when Terraform code contains syntax errors or other issues unrelated to input validation (e.g., duplicate attributes, circular references, invalid locals). The "valid" designation means "valid for input validation purposes" not "syntactically valid Terraform/OpenTofu code".
Applied to files:
test/integration_test.gotest/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcl
📚 Learning: 2025-02-10T23:20:04.295Z
Learnt from: yhakbar
Repo: gruntwork-io/terragrunt PR: 3868
File: docs-starlight/patches/@astrojs%[email protected]:33-33
Timestamp: 2025-02-10T23:20:04.295Z
Learning: In Terragrunt projects, all `.hcl` files can be assumed to be Terragrunt configurations by default, with specific exceptions like `.terraform.lock.hcl` that need explicit handling.
Applied to files:
test/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcl
🧬 Code graph analysis (1)
test/integration_test.go (2)
test/helpers/package.go (6)
CleanupTerraformFolder(882-889)CopyEnvironment(89-105)RunTerragrunt(979-983)RunTerragruntCommand(965-969)LogBufferContentsLineByLine(985-993)TerraformOutput(83-87)util/file.go (1)
JoinPath(626-628)
🔇 Additional comments (4)
docs-starlight/src/content/docs/04-reference/01-hcl/04-functions.mdx (1)
907-950: Documentation structure and content are well done.The section follows the established pattern from the non-cached variant, provides clear examples, and includes important caveats about caching semantics. The guidance on when to use each function variant is helpful.
test/fixtures/read-config-with-cache/from_dependency/main.tf (1)
1-2: Fixture variable/output passthrough looks correct
variable "bar"and the correspondingoutput "bar"cleanly expose the input for use by Terragrunt; no issues from a test-fixture perspective.test/fixtures/read-config-with-cache/from_dependency/terragrunt.hcl (1)
1-7: Dependency wiring and inputs mapping are consistentThe dependency block and
inputs.bar = dependency.dep.outputs.foofollow the established pattern for dependency-based fixtures and should work as intended.test/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcl (1)
1-18: Cached read-config fixture mirrors existing original-dir coverageUsing
read_terragrunt_config_with_cache("foo/bar.hcl")and wiring both local and dependency outputs intoinputsmatches the original (non-cache) fixture’s intent; this is a good basis for verifying that caching preservesterragrunt_dir/original_terragrunt_dirsemantics.
jkarkoszka
force-pushed
the
feat/cache-for-read-terragrunt-config
branch
from
81f3389 to
72da811
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
test/integration_test.go (3)
2603-2652: Cached dependency test correctly mirrors non-cached behavior and tightens type checking
TestReadTerragruntConfigWithCacheWithDependencyis structurally identical to the non-cached test but points at the cached fixture root, ensuring semantic parity, and the explicitfloat64check foroutputs["number"]avoids panics from unexpected JSON types. This is a solid, robust integration test for the cached variant.If you find the duplication with
TestReadTerragruntConfigWithDependencystarts to drift over time, consider a small shared helper that takes the fixture root (cached vs non-cached) as a parameter to keep them in lockstep.
2708-2796: OriginalTerragruntDir cached test accurately encodes the intentional semantic difference
TestReadTerragruntConfigWithCacheWithOriginalTerragruntDirclosely follows the non-cached test but intentionally expectsrunAllDepOutputs["bar_original_terragrunt_dir"] == rootPathAbs, validating thatread_terragrunt_config_with_cachereturns the first caller’s context on subsequent reads of the same config path. This lines up with the documented design trade-off for the cached helper. Consider adding a brief comment above the final assertion to spell out that this divergence from the non-cached test is deliberate to guard against future “fixes.”Based on learnings
2798-2930: Full cached read-config test duplicates non-cached assertions to prove parity
TestReadTerragruntConfigWithCacheFullis effectively a copy of the non-cached “full” test pointed attestFixtureReadConfigWithCache/full, asserting every field (simple scalars, JSON-stringified maps, generate/remote_state/terraform blocks, and hooks). This is an appropriate level of depth to guarantee that caching does not alter the observable config shape.If maintaining two nearly identical “full” tests becomes cumbersome, you could extract a helper that takes the fixture root and maybe the function-under-test name for log messages, and invoke it from both cached and non-cached tests.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (23)
config/config_helpers.go(3 hunks)config/context.go(1 hunks)docs-starlight/src/content/docs/04-reference/01-hcl/04-functions.mdx(1 hunks)docs-starlight/src/data/commands/find.mdx(1 hunks)docs-starlight/src/data/flags/find-reading.mdx(1 hunks)test/fixtures/read-config-with-cache/from_dependency/dep/main.tf(1 hunks)test/fixtures/read-config-with-cache/from_dependency/dep/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/from_dependency/dep/vars.hcl(1 hunks)test/fixtures/read-config-with-cache/from_dependency/main.tf(1 hunks)test/fixtures/read-config-with-cache/from_dependency/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/full/main.tf(1 hunks)test/fixtures/read-config-with-cache/full/source.hcl(1 hunks)test/fixtures/read-config-with-cache/full/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_default/main.tf(1 hunks)test/fixtures/read-config-with-cache/with_default/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_dependency/dep/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_dependency/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/main.tf(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/foo/bar.hcl(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/main.tf(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcl(1 hunks)test/integration_test.go(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (17)
- docs-starlight/src/data/flags/find-reading.mdx
- test/fixtures/read-config-with-cache/from_dependency/dep/terragrunt.hcl
- docs-starlight/src/data/commands/find.mdx
- test/fixtures/read-config-with-cache/from_dependency/dep/main.tf
- test/fixtures/read-config-with-cache/with_dependency/dep/terragrunt.hcl
- config/context.go
- test/fixtures/read-config-with-cache/with_dependency/terragrunt.hcl
- test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/main.tf
- test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/terragrunt.hcl
- test/fixtures/read-config-with-cache/from_dependency/terragrunt.hcl
- test/fixtures/read-config-with-cache/with_original_terragrunt_dir/main.tf
- test/fixtures/read-config-with-cache/with_default/main.tf
- test/fixtures/read-config-with-cache/from_dependency/main.tf
- docs-starlight/src/content/docs/04-reference/01-hcl/04-functions.mdx
- config/config_helpers.go
- test/fixtures/read-config-with-cache/from_dependency/dep/vars.hcl
- test/fixtures/read-config-with-cache/full/source.hcl
🧰 Additional context used
📓 Path-based instructions (1)
**/*.go
⚙️ CodeRabbit configuration file
Review the Go code for quality and correctness. Make sure that the Go code follows best practices, is performant, and is easy to understand and maintain.
Files:
test/integration_test.go
🧠 Learnings (4)
📓 Common learnings
Learnt from: jkarkoszka
Repo: gruntwork-io/terragrunt PR: 5116
File: test/integration_test.go:2785-2795
Timestamp: 2025-11-18T09:54:06.008Z
Learning: In Terragrunt, `read_terragrunt_config_with_cache` intentionally has different semantics than `read_terragrunt_config`: it caches the first parse result (including context like `original_terragrunt_dir`) and returns that cached value on subsequent calls to the same file, even if the calling context has changed. This is a performance optimization that trades context accuracy for speed. Tests showing different assertions between cached and non-cached variants for context-dependent values like `original_terragrunt_dir` are expected and correct.
📚 Learning: 2025-11-18T09:54:06.008Z
Learnt from: jkarkoszka
Repo: gruntwork-io/terragrunt PR: 5116
File: test/integration_test.go:2785-2795
Timestamp: 2025-11-18T09:54:06.008Z
Learning: In Terragrunt, `read_terragrunt_config_with_cache` intentionally has different semantics than `read_terragrunt_config`: it caches the first parse result (including context like `original_terragrunt_dir`) and returns that cached value on subsequent calls to the same file, even if the calling context has changed. This is a performance optimization that trades context accuracy for speed. Tests showing different assertions between cached and non-cached variants for context-dependent values like `original_terragrunt_dir` are expected and correct.
Applied to files:
test/fixtures/read-config-with-cache/full/terragrunt.hcltest/integration_test.gotest/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcltest/fixtures/read-config-with-cache/with_default/terragrunt.hcltest/fixtures/read-config-with-cache/with_original_terragrunt_dir/foo/bar.hcl
📚 Learning: 2025-11-03T04:40:01.000Z
Learnt from: ThisGuyCodes
Repo: gruntwork-io/terragrunt PR: 5041
File: test/fixtures/hclvalidate/valid/duplicate-attributes-in-required-providers/main.tf:2-7
Timestamp: 2025-11-03T04:40:01.000Z
Learning: In the terragrunt repository, test fixtures under test/fixtures/hclvalidate/valid/ are intentionally testing that INPUT validation succeeds even when Terraform code contains syntax errors or other issues unrelated to input validation (e.g., duplicate attributes, circular references, invalid locals). The "valid" designation means "valid for input validation purposes" not "syntactically valid Terraform/OpenTofu code".
Applied to files:
test/fixtures/read-config-with-cache/full/terragrunt.hcltest/integration_test.gotest/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcltest/fixtures/read-config-with-cache/with_default/terragrunt.hcltest/fixtures/read-config-with-cache/with_original_terragrunt_dir/foo/bar.hcl
📚 Learning: 2025-02-10T23:20:04.295Z
Learnt from: yhakbar
Repo: gruntwork-io/terragrunt PR: 3868
File: docs-starlight/patches/@astrojs%[email protected]:33-33
Timestamp: 2025-02-10T23:20:04.295Z
Learning: In Terragrunt projects, all `.hcl` files can be assumed to be Terragrunt configurations by default, with specific exceptions like `.terraform.lock.hcl` that need explicit handling.
Applied to files:
test/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcltest/fixtures/read-config-with-cache/with_default/terragrunt.hcltest/fixtures/read-config-with-cache/with_original_terragrunt_dir/foo/bar.hcl
🧬 Code graph analysis (1)
test/integration_test.go (2)
test/helpers/package.go (6)
CleanupTerraformFolder(882-889)CopyEnvironment(89-105)RunTerragrunt(979-983)RunTerragruntCommand(965-969)LogBufferContentsLineByLine(985-993)TerraformOutput(83-87)util/file.go (1)
JoinPath(626-628)
🔇 Additional comments (7)
test/fixtures/read-config-with-cache/with_default/terragrunt.hcl (1)
1-5: Fixture correctly exercises default value behavior for cached read-configUsing
read_terragrunt_config_with_cachewith a non-existent file and a{data = "default value"}default, then wiringinputs = local.config_does_not_exist, matches the non-cached fixture semantics and cleanly tests the default path. No issues from a Terragrunt/HCL standpoint.test/fixtures/read-config-with-cache/with_original_terragrunt_dir/foo/bar.hcl (1)
1-4: Locals for dir/original_dir look correct and minimalThe fixture cleanly exposes
get_terragrunt_dir()andget_original_terragrunt_dir()for downstream assertions in the cached tests; structure and naming are consistent with the existing (non-cached) fixture.test/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcl (1)
1-18: Root fixture correctly wires cached config and dependency outputsThe
locals.bar = read_terragrunt_config_with_cache("foo/bar.hcl")plus thedependency "dep"and the aggregatedinputsmap mirror the non-cached setup while isolating the change to the cached function, which is exactly what the integration tests rely on. Paths and input keys look consistent with the assertions.test/fixtures/read-config-with-cache/full/terragrunt.hcl (1)
1-19: Full cached fixture mirrors non-cached semantics and matches main.tfThe
config = read_terragrunt_config_with_cache("${get_terragrunt_dir()}/source.hcl")and the subsequentinputsmap cleanly forward all relevant attributes (includingterraformtg, version constraints, hooks, etc.) to Terraform. Names line up with the variables/outputs infull/main.tfand with the expectations in the full cached test.test/integration_test.go (2)
37-125: NewtestFixtureReadConfigWithCacheconstant follows existing fixture naming patternAdding
testFixtureReadConfigWithCache = "fixtures/read-config-with-cache"keeps fixture roots consistent with the long-standingtestFixtureReadConfigstyle and makes the cached tests easy to locate and reason about.
2654-2706: From-dependency and default-path cached tests are aligned with existing coverageBoth
TestReadTerragruntConfigWithCacheFromDependencyandTestReadTerragruntConfigWithCacheWithDefaultreuse the established patterns from the non-cached equivalents—same command shapes, same output assertions—while just swapping totestFixtureReadConfigWithCache. This cleanly validates that the cached variant behaves identically for dependency wiring and default-value fallback on missing configs.test/fixtures/read-config-with-cache/full/main.tf (1)
1-38: Terraform passthrough variables/outputs correctly mirror Terragrunt inputsEach
variable/outputpair directly mirrors the keys set infull/terragrunt.hcl(localstg,generate,remote_state,terraformtg, etc.), making the integration test assertions straightforward and ensuring the cached config content is fully observable. This fixture is simple and correct for its purpose.
jkarkoszka
changed the title
jkarkoszka
force-pushed
the
feat/cache-for-read-terragrunt-config
branch
from
72da811 to
e90b735
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
config/config_helpers.go (1)
54-55: Cached function implementation matches non-cached semantics, with clear cache keying
- The new
FuncNameReadTerragruntConfigWithCacheconstant and registration interragruntFunctionsare straightforward and mirror the existingread_terragrunt_configwiring.readTerragruntConfigAsFuncImplWithCache:
- Validates parameter count correctly and reports
WrongNumberOfParamsErrorwith the new function name.- Uses the same parameter conventions (1 required string + optional dynamic) as the non-cached variant.
- Normalizes the target path via
getCleanedTargetConfigPathand uses that plushasDefault=%tin the cache key, which avoids mixing “no-default” and “with-default” calls for the same config.- Logs cache hits/misses at debug level, which will be useful in diagnosing behavior without being noisy at normal levels.
- Delegates the actual parsing to
ParseTerragruntConfig, so the first call behaves identically to the non-cached function and only subsequent calls benefit from caching.Minor nit: the new comment reads “used to for calling”; consider tightening it to “used to call” for clarity:
-// Create a cty Function that can be used to for calling read_terragrunt_config_with_cache. +// Create a cty Function that can be used to call read_terragrunt_config_with_cache.Otherwise the implementation looks sound and aligns with the documented “first parse wins” semantics for context-sensitive fields like
original_terragrunt_dir. Based on learnings.Also applies to: 152-160, 823-878
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (23)
config/config_helpers.go(3 hunks)config/context.go(1 hunks)docs-starlight/src/content/docs/04-reference/01-hcl/04-functions.mdx(1 hunks)docs-starlight/src/data/commands/find.mdx(1 hunks)docs-starlight/src/data/flags/find-reading.mdx(1 hunks)test/fixtures/read-config-with-cache/from_dependency/dep/main.tf(1 hunks)test/fixtures/read-config-with-cache/from_dependency/dep/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/from_dependency/dep/vars.hcl(1 hunks)test/fixtures/read-config-with-cache/from_dependency/main.tf(1 hunks)test/fixtures/read-config-with-cache/from_dependency/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/full/main.tf(1 hunks)test/fixtures/read-config-with-cache/full/source.hcl(1 hunks)test/fixtures/read-config-with-cache/full/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_default/main.tf(1 hunks)test/fixtures/read-config-with-cache/with_default/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_dependency/dep/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_dependency/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/main.tf(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/terragrunt.hcl(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/foo/bar.hcl(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/main.tf(1 hunks)test/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcl(1 hunks)test/integration_test.go(2 hunks)
✅ Files skipped from review due to trivial changes (1)
- docs-starlight/src/content/docs/04-reference/01-hcl/04-functions.mdx
🚧 Files skipped from review as they are similar to previous changes (17)
- test/fixtures/read-config-with-cache/from_dependency/terragrunt.hcl
- test/fixtures/read-config-with-cache/with_dependency/dep/terragrunt.hcl
- test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/terragrunt.hcl
- test/fixtures/read-config-with-cache/with_original_terragrunt_dir/dep/main.tf
- test/fixtures/read-config-with-cache/from_dependency/dep/main.tf
- test/fixtures/read-config-with-cache/from_dependency/main.tf
- test/fixtures/read-config-with-cache/from_dependency/dep/vars.hcl
- test/fixtures/read-config-with-cache/with_original_terragrunt_dir/terragrunt.hcl
- docs-starlight/src/data/flags/find-reading.mdx
- test/fixtures/read-config-with-cache/full/source.hcl
- docs-starlight/src/data/commands/find.mdx
- test/fixtures/read-config-with-cache/with_original_terragrunt_dir/foo/bar.hcl
- test/fixtures/read-config-with-cache/full/main.tf
- test/fixtures/read-config-with-cache/full/terragrunt.hcl
- test/fixtures/read-config-with-cache/with_default/terragrunt.hcl
- test/fixtures/read-config-with-cache/with_dependency/terragrunt.hcl
- test/fixtures/read-config-with-cache/from_dependency/dep/terragrunt.hcl
🧰 Additional context used
📓 Path-based instructions (1)
**/*.go
⚙️ CodeRabbit configuration file
Review the Go code for quality and correctness. Make sure that the Go code follows best practices, is performant, and is easy to understand and maintain.
Files:
config/context.gotest/integration_test.goconfig/config_helpers.go
🧠 Learnings (6)
📓 Common learnings
Learnt from: jkarkoszka
Repo: gruntwork-io/terragrunt PR: 5116
File: test/integration_test.go:2785-2795
Timestamp: 2025-11-18T09:54:06.008Z
Learning: In Terragrunt, `read_terragrunt_config_with_cache` intentionally has different semantics than `read_terragrunt_config`: it caches the first parse result (including context like `original_terragrunt_dir`) and returns that cached value on subsequent calls to the same file, even if the calling context has changed. This is a performance optimization that trades context accuracy for speed. Tests showing different assertions between cached and non-cached variants for context-dependent values like `original_terragrunt_dir` are expected and correct.
📚 Learning: 2025-11-18T09:54:06.008Z
Learnt from: jkarkoszka
Repo: gruntwork-io/terragrunt PR: 5116
File: test/integration_test.go:2785-2795
Timestamp: 2025-11-18T09:54:06.008Z
Learning: In Terragrunt, `read_terragrunt_config_with_cache` intentionally has different semantics than `read_terragrunt_config`: it caches the first parse result (including context like `original_terragrunt_dir`) and returns that cached value on subsequent calls to the same file, even if the calling context has changed. This is a performance optimization that trades context accuracy for speed. Tests showing different assertions between cached and non-cached variants for context-dependent values like `original_terragrunt_dir` are expected and correct.
Applied to files:
config/context.gotest/fixtures/read-config-with-cache/with_original_terragrunt_dir/main.tftest/integration_test.goconfig/config_helpers.go
📚 Learning: 2025-11-03T04:40:01.000Z
Learnt from: ThisGuyCodes
Repo: gruntwork-io/terragrunt PR: 5041
File: test/fixtures/hclvalidate/valid/duplicate-attributes-in-required-providers/main.tf:2-7
Timestamp: 2025-11-03T04:40:01.000Z
Learning: In the terragrunt repository, test fixtures under test/fixtures/hclvalidate/valid/ are intentionally testing that INPUT validation succeeds even when Terraform code contains syntax errors or other issues unrelated to input validation (e.g., duplicate attributes, circular references, invalid locals). The "valid" designation means "valid for input validation purposes" not "syntactically valid Terraform/OpenTofu code".
Applied to files:
test/fixtures/read-config-with-cache/with_original_terragrunt_dir/main.tftest/integration_test.go
📚 Learning: 2025-02-10T13:36:19.542Z
Learnt from: levkohimins
Repo: gruntwork-io/terragrunt PR: 3723
File: cli/commands/stack/action.go:160-160
Timestamp: 2025-02-10T13:36:19.542Z
Learning: The project uses a custom error package `github.com/gruntwork-io/terragrunt/internal/errors` which provides similar functionality to `fmt.Errorf` but includes stack traces. Prefer using this package's error functions (e.g., `errors.Errorf`, `errors.New`) over the standard library's error handling.
Applied to files:
config/config_helpers.go
📚 Learning: 2025-09-10T04:41:35.652Z
Learnt from: jorhett
Repo: gruntwork-io/terragrunt PR: 1234
File: errors/multierror.go:35-39
Timestamp: 2025-09-10T04:41:35.652Z
Learning: In Terragrunt's MultiError.Error() method, checking for the exact string "exit status 2" and returning it directly is not a brittle hack but a semantic fix. Terraform's --detailed-exitcode flag uses exit code 2 to mean "plan succeeded with changes" (not an error), so when multiple modules return this status, it should not be wrapped in "Hit multiple errors:" formatting as that misrepresents successful operations as errors.
Applied to files:
config/config_helpers.go
📚 Learning: 2025-04-17T13:02:28.098Z
Learnt from: yhakbar
Repo: gruntwork-io/terragrunt PR: 4169
File: cli/commands/hcl/validate/cli.go:29-60
Timestamp: 2025-04-17T13:02:28.098Z
Learning: Avoid shadowing imported packages with local variables in Go code, such as using a variable named `flags` when the `github.com/gruntwork-io/terragrunt/cli/flags` package is imported. Use more specific variable names like `flagSet` instead.
Applied to files:
config/config_helpers.go
🧬 Code graph analysis (3)
config/context.go (1)
internal/cache/cache.go (1)
NewCache(24-30)
test/integration_test.go (2)
test/helpers/package.go (6)
CleanupTerraformFolder(882-889)CopyEnvironment(89-105)RunTerragrunt(979-983)RunTerragruntCommand(965-969)LogBufferContentsLineByLine(985-993)TerraformOutput(83-87)util/file.go (1)
JoinPath(626-628)
config/config_helpers.go (5)
config/parsing_context.go (1)
ParsingContext(17-60)config/errors.go (1)
WrongNumberOfParamsError(89-93)options/options.go (1)
TerragruntOptions(99-329)internal/cache/cache.go (1)
ContextCache(121-128)config/context.go (1)
ReadTerragruntConfigCacheContextKey(17-17)
🔇 Additional comments (4)
test/fixtures/read-config-with-cache/with_default/main.tf (1)
1-4: Fixture wiring looks correctSimple pass-through of
var.datatooutput "data"is exactly what this fixture needs; no issues.test/fixtures/read-config-with-cache/with_original_terragrunt_dir/main.tf (1)
1-47: Variables/outputs align with integration test expectationsThe string variables and 1:1 outputs for the various
terragrunt_dir/original_terragrunt_dircombinations match how the integration test reads them and cleanly separate root vs dep vs dep_bar cases; looks good.config/context.go (1)
8-9: New read-config cache context key and wiring look consistent
- Adding
ReadTerragruntConfigCacheContextKeybetween the existing cache keys keeps theconfigKey/iotasequence coherent.- Initializing
cache.NewCache[cty.Value](readTerragruntConfigCacheName)inWithConfigValuesmirrors the other caches and ensures the typed cache is always present whenContextCacheis used.- The dedicated
readTerragruntConfigCacheNamestring is a nice improvement over relying on%vof the key.No functional or concurrency concerns here.
Also applies to: 14-27, 32-36
test/integration_test.go (1)
98-99: Cached read-config tests mirror existing coverage and correctly encode the new semantics
testFixtureReadConfigWithCacheconstant cleanly parallelstestFixtureReadConfigand isolates the new fixtures.- The four “standard” tests (
WithCacheWithDependency,WithCacheFromDependency,WithCacheWithDefault,WithCacheFull) closely mirror the existing non-cached tests, which is ideal for regression coverage and validates that the cached variant is behaviorally equivalent in the common cases.TestReadTerragruntConfigWithCacheWithOriginalTerragruntDir:
- Is structured identically to the non-cached version, but intentionally asserts
bar_original_terragrunt_dir == rootPathAbsin the finalrun --allscenario, documenting the “first caller’s context is cached” behavior fororiginal_terragrunt_dir.- This matches the intended trade-off for
read_terragrunt_config_with_cache(performance over per-call context accuracy) and makes the difference explicit and test-backed. Based on learnings.The additional type assertion guard for the numeric output in the dependency test is a nice robustness touch. Overall these tests look solid and aligned with the new function’s contract.
Also applies to: 2603-2930
|
This is a great feature that would help us improve the runtime of our pipelines. |
2 participants
Description
This PR introduces a new function,
read_terragrunt_config_with_cache, which provides optional, opt-in caching for Terragrunt config parsing. In large Terragrunt infrastructures, shared configuration files (e.g.,global-accounts.hcl,global-modules.hcl,global-vars.hcl, etc.) may be parsed thousands of times during a singlerun-alloperation. This causes a significant amount of redundant HCL parsing and increases execution time, especially in complex multi-account AWS environments.This work implements one of the option described in #4896, which outlines the motivation, performance impact, and proposed design for optional caching of repeated config reads.
The newly added function behaves like
read_terragrunt_configbut stores the parsed result in an in-memory cache keyed by absolute file path. Subsequent calls for the same file return the cached value, eliminating redundant parsing.Key points
Opt-in: Existing Terragrunt behavior remains unchanged.
Performance improvement: Real-world project tested — 1956 redundant parses reduced to a single parse, producing a ~35% reduction in pipeline runtime.
Safe by default: Caching is not enabled automatically due to compatibility considerations with get_original_terragrunt_dir().
The caching implementation is isolated to the new function (read_terragrunt_config_with_cache) to avoid breaking existing setups.
Changes in this PR
Introduces read_terragrunt_config_with_cache(path)
Implements an internal global in-memory cache keyed by canonical absolute path
Adds tests covering standard caching cases
Documents behavior, caveats, and usage
Ensures the default function (read_terragrunt_config) remains unchanged
This feature enables teams with large infrastructures to materially speed up Terragrunt operations without modifying existing configurations.
TODOs
Release Notes (draft)
Added
Migration Guide
Terragrunt now provides an optional caching mechanism for repeated config parsing via a new function:
read_terragrunt_config_with_cache(path).
Caching is disabled by default to preserve existing behavior and avoid compatibility issues for users relying on evaluation-context-dependent functions.
Who should migrate?
Users with:
If your infrastructure parses the same *.hcl configuration files hundreds or thousands of times, you are likely to benefit.
How to migrate
Simply replace:
with:
No other changes are required.
Summary by CodeRabbit
New Features
Documentation
Tests