Skip to content

Provider caching fails to prevent excessive disk space consumption on Windows #5158

@Michagogo

Description

@Michagogo

Describe the bug

When using the provider cache feature on Windows, it seems that the linking functionality doesn’t work properly and each module has the provider copied into its .terragrunt-cache directory, taking up large amounts of space (e.g. in the case of the aws provider, this is >700MB per module).
This issue seems to be unique to Windows — when running Terragrunt on the exact same configuration within WSL everything appears to function as expected.

Steps To Reproduce

  1. $Env:TG_PROVIDER_CACHE=1
  2. Run (apply, plan, etc. — or even just init) a configuration containing several (e.g. 5) modules that use the same provider of non-trivial size.
  3. Observe that the output printed to the console does show the cache operating, with the messages about the cache server starting, each module having its provider cached, etc.
  4. Navigate the .terragrunt-cache directory of each module, and observe that each one contains a copy of the provider, as opposed to being a link to the centrally cached provider as advertised.
  5. To further confirm that the files are discrete copies (rather than simply being a case of Explorer not showing links well), note free disk space on the drive in question.
  6. Delete the per-module caches (e.g. by running find -name .terragrunt-cache -type d -exec rm -rf {} \; in the root of the configuration directory).
  7. Check the free disk space once more, and observe that the amount freed is roughly consistent with the size of the provider multiplied by the number of modules, rather than being a negligible amount (i.e. not much more than the size of the module source itself).

Expected behavior

Once the provider is downloaded and cached into the central location, any modules using the provider should not take up any more disk space (and in fact even the first one should not be taking up any space in the module’s directory itself).

Versions

  • Terragrunt version: 0.93.x (as well as a much older version when we first encountered this issue a while back)
  • OpenTofu/Terraform version: Presumably not relevant, as the behavior in question is TG’s
  • Environment details (Ubuntu 20.04, Windows 10, etc.): Windows 11

Additional context

Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions