Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

terragrunt apply-all specific plan fails #1271

Closed
ibacalu opened this issue Jul 30, 2020 · 9 comments · Fixed by #1740 · May be fixed by #1429
Closed

terragrunt apply-all specific plan fails #1271

ibacalu opened this issue Jul 30, 2020 · 9 comments · Fixed by #1740 · May be fixed by #1429
Labels
bug Something isn't working enhancement New feature or request

Comments

@ibacalu
Copy link

ibacalu commented Jul 30, 2020

This is not a new issue, and I can't believe it wasn't fixed so far.
Running terragrunt apply-all blindly is the perfect recipe for disaster.
In order to make this more acceptable for automation, we should at least allow for a reviewed plan to be applied.
The issue resides with forced silent arguments -input=false and -auto-approve.
These arguments are added after any extra_arguments, which generates invalid terraform command
At least allow us to override your forced arguments.

# Generated command
terraform apply reviewed-plan.tfplan -input=false -auto-approve

...
Too many command line arguments. Configuration path expected.
@ibacalu
Copy link
Author

ibacalu commented Jul 30, 2020

this bug is also mentioned on issue #848 however that ticket is closed and there doesn't seem to be any fix or workaround in sight.

@ibacalu
Copy link
Author

ibacalu commented Jul 30, 2020

there's also a good suggestion here: #454 (comment)

@yorinasub17
Copy link
Contributor

This is actually not recommended to do, and in fact, is impossible due to terraform limitations (you can't have a plan that spans multiple state files). See #1127 (comment) for additional commentary on the -all commands.

@yorinasub17 yorinasub17 added enhancement New feature or request needs design bug Something isn't working labels Jul 31, 2020
@yorinasub17
Copy link
Contributor

For plan-all, I forgot to mention that we want to solve this eventually, but currently we need a few more design cycles to come up with a proper solution.


As far as fixing the specific error with the forced args, we could certainly get that fixed, but it doesn't resolve the fundamental issue with using reviewed plans on apply-all (which is an anti-pattern).

@ibacalu
Copy link
Author

ibacalu commented Jul 31, 2020

For plan-all, I forgot to mention that we want to solve this eventually, but currently we need a few more design cycles to come up with a proper solution.

As far as fixing the specific error with the forced args, we could certainly get that fixed, but it doesn't resolve the fundamental issue with using reviewed plans on apply-all (which is an anti-pattern).

Hello,

It's not anti-pattern. I am not suggesting to use one single plan to apply.
Check out the code below. All works except this issue with -input=false and -auto-approve forcibly added at the end of the generated command.

terraform {
  before_hook "generate_plan_path" {
    commands     = ["plan"]
    execute      = ["mkdir", "-p", "${get_parent_terragrunt_dir()}/plans/${path_relative_to_include()}"]
  }

  extra_arguments "plan_args" {
    commands = [
      "plan"
    ]

    arguments = [
      "-out", 
      "${get_parent_terragrunt_dir()}/plans/${path_relative_to_include()}/tf.p"
    ]
  }
  
  extra_arguments "apply_args" {
    commands = [
      "apply"
    ]

    arguments = [
      "${get_parent_terragrunt_dir()}/plans/${path_relative_to_include()}/tf.p",
    ]
  }
}

@yorinasub17
Copy link
Contributor

To clarify, we (Gruntwork) consider it an antipattern to use plan-all => apply-all on a day to day basis, because plan-all is fundamentally broken right now when you have cross module dependencies. If you have no cross module dependencies, then, yes plan-all works, but I suspect most sufficiently complex deployments will have cross module dependencies.

All I was trying to say is that there is much more than just having the single module plan files work with extra_arguments if you want the actual experience of using plan files with xxx-all commands.

In any case, as mentioned above, a PR to fix this particular issue with injected args is certainly welcome. I think this could be fixed by updating the logic in this function to look for positional args in argsToInsert and insert them at the end.

@ibacalu
Copy link
Author

ibacalu commented Jul 31, 2020

@yorinasub17 I think i understand what you mean now. It was a bit confusing. So apply-all doesn't actually run apply on plans shown when you do plan-all (because of mocked resources) but instead it re-runs the plan right before applying.
I didn't look at the code, I am just guessing here.
It's sad, because it would make so much sense. At least to have some way of getting the dependency graph and based on that to configure the CI with automatic sequential plan and apply on specific resources.

@yorinasub17
Copy link
Contributor

So apply-all doesn't actually run apply on plans shown when you do plan-all (because of mocked resources) but instead it re-runs the plan right before applying.

Yup this is basically correct. apply-all works because it runs the apply in dependency order, so the upstream module changes are reflected by the time the downstream module apply happens. When you introduce plan files, this all breaks down, especially if you are using mock outputs, since the downstream plan will be based on garbage input.

@uptickmetachu
Copy link

Just bumping this one.

I'm currently trying to make a CI pipeline using terragrunt and I'm running into this as well.

The workflow I had in mind was:

  1. Make a PR with changes to multiple modules
  2. CI generates plans using terragrunt plan-all and storing the plan in each module folder.
  3. CI filters the plans to only those with changes and without parent dependencies with changes.
  4. User verifies plan outputs and makes a PR comment "Apply all (perhaps with a hash representing a set of plans to apply)
  5. CI runs terragrunt apply-all with a --terragrunt-include-dir {} for each plan, thus restricting the plans being applied. Extra Args will also be used like above to run terraform apply with a plan for each module.
  6. Repeat 2-5 until nothing else is planned and everything is applied.

Currently this isn't possible because specifying a plan with terragrunt apply-all does not work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
3 participants