Estimate provisioning time

[magodo]

Current Terraform Version

v0.15

Use-cases

Terraform command like plan/apply/destroy can tell users an estimation of the time needed to finish the operation, so that users can have a sense on how long this provision will take.

Proposal

Add a new step in the Plan/Apply/Destroy Graph, during which the TF will query the provider(s) for an estimation of the current execution plan. Given the plan, the provider can do a check on the input properties of each resource and whether this resource is newly created or is an update, produce an estimation on the time needed, per resource. As a fallback, if there is no logic provided by the provider, the Timeout settings for the corresponding operation is returned. Then the TF can collect each resource's estimation, plus the provisioning order and relationships, with parallelism taken into consideration, generate a final provision estimation.

[apparentlymart]

Hi @magodo! Thanks for sharing this use-case.

It's definitely an interesting idea to explore! I want to be up front that I don't expect we'll be able to tackle this in the near future because it's a pretty cross-cutting problem that will require lots of coordination with provider developers to design and add the relevant information for it to be useful, and even then will need to think about how best to estimate concurrency for as you say, because unexpectedly fast or slow operations could potentially change the outcome in a material way.

I wonder about a simpler version of this idea for a first pass where we might allow specific actions returned from a provider to include some rough indication of duration (perhaps a real time estimate, or perhaps just a rough order of magnitude) and then annotate that specific item in the plan with the estimate, and then try to encourage provider developers to add those annotations just for operations that typically take more than a few seconds to run, in order to give a prior warning that a particular apply is likely to take longer than it might appear just from counting how many operations are planned. That would potentially allow for a more gradual and focused addition of these, potentially growing to a more comprehensive solution over time if there are enough annotations in the providers to make the result useful.

[nitrocode]

I'd personally love something like apt. This would allow us to estimate when a module can be fully applied/destroyed without having to guess or without having to run and forget about the apply. Some applies take a while like msk, eks, etc and being able to know which resource out of the total resources is a good start.

[1 / 100] module.ec2_client_vpn.aws_ec2_client_vpn_route.default[8]: Destroying... 
[2 / 100] module.ec2_client_vpn.aws_ec2_client_vpn_route.default[9]: Destroying... 
[1 / 100] module.ec2_client_vpn.aws_ec2_client_vpn_route.default[8]: Destruction complete
[2 / 100] ...
...
[40 / 100] ...
[start-time 40% ] |############# . . . . . . . . . . . . .| [estimated completion time]

In the future, if providers allow time estimates per resource, that would be even better. For now, I'd be happy with to see which resource is being applied out of how many resources.

Ref

• https://mdk.fr/blog/how-apt-does-its-fancy-progress-bar.html
• https://www.bbc.com/future/article/20191205-why-progress-bars-can-make-you-feel-better

[RobbertDM]

I missed this feature today when refreshing 60000 resources :D

[magodo]

I've just created a tool to display the progress (and more): https://github.com/magodo/pipeform