Option to disable creation of assets for local modules

[coen-hyde]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

We would like the option to disable the creation of assets for local modules behavior introduced in #1476. This new way means that for a cdktf application to run in Terraform Cloud with the latest changes to a module, someone needs to execute cdktf synth and commit the generated changes. We were previously triggering runs on cdktf app workspaces if a module it uses changed via the VCS workflow. In addition we do not really want to commit into source an asset directory that has the same module copied dozens of times.

As I understand it, this change was made to improve compatibility with Terraform Cloud. The old way worked for us well in Terraform Cloud. Is there something we are missing? Maybe there is an alternative flow that we are not aware of?

[jsteinich]

There are a couple motivations for local modules to be treated as assets:

• When invoking Terraform Cloud from cdktf, any modules used need to be accessible. Packing as assets is a convenient way to ensure this is true.
• Using assets centralizes the handling of relative paths (very common when using local modules) and results in a consistent experience.
• Packaging everything together makes it very clear what source resulted in the synthesized output (More of an argument that the AWS CDK makes since lookups are very common in Terraform).

I do have a few questions / comments about your workflow to help understand the best course of action:

• Are the modules in the same repository as the cdktf app?
• Is it possible to just trigger cdktf synth as part of the VCS workflow? Not having to run cdktf synth locally is a nicer flow from my experience.
• What's the motivation for using a local module rather than just building into the cdktf app?

[coen-hyde]

Yeah I get the motivation, it makes sense for a lot of workflows, just not ours.

Are the modules in the same repository as the cdktf app?

They are. We have a mono repo for our infrastructure that includes our modules, workspaces and cdktf apps.

Is it possible to just trigger cdktf synth as part of the VCS workflow? Not having to run cdktf synth locally is a nicer flow from my experience.

I guess we could. But this will still make our git history kind of gross.

What's the motivation for using a local module rather than just building into the cdktf app?

Most of our infrastructure is regular Terraform. These modules are just regular terraform modules which are used in other workspaces as well as the cdktf apps. We would prefer not to duplicate logic. Also writing Terraform in cdktf has a higher barrier for entry, which means fewer people have the expertise to make changes. So for these reason we try to do what we can in Terraform before reaching for cdktf.

[jsteinich]

Is it possible to just trigger cdktf synth as part of the VCS workflow? Not having to run cdktf synth locally is a nicer flow from my experience.

I guess we could. But this will still make our git history kind of gross.

The idea would be that the synth output (including assets) then doesn't need to be committed.
If that isn't possible, then I think an option to disable automatic options could be a good compromise.

Also writing Terraform in cdktf has a higher barrier for entry

I'm interested to hear more about why you think this way. Ideally the barrier is actually lower (at least for developers).

[coen-hyde]

The idea would be that the synth output (including assets) then doesn't need to be committed.
If that isn't possible, then I think an option to disable automatic options could be a good compromise.

I don't think this is possible with the VCS workflow in Terraform Cloud. As far as i know you can't run a pre-processing script to do something like this.

I'm interested to hear more about why you think this way. Ideally the barrier is actually lower (at least for developers).

cdktf is a layer of abstraction on top of Terraform but it is a rather transparent layer. To write cdktf you need to know how Terraform functions and how to code. That is two knowledge dependencies. Where as Terraform is only one. In an environment where we are trying to encourage people to use Infrastructure as Code instead of console clicking, reducing the barrier to entry is important.