Unable to apply changes when using an auto-generated Atlantis config

[snorlaX-sleeps]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Overview of the Issue

Unsure if this is a bug, a "looking for information on how to", or a misconfiguration:

We have developed a script to create an Atlantis autogenerated config file when a PR is created, which parses our repo and generates an atlantis.yaml file.
The file is correctly generated for a repo when the PR is initially opened (file is uploaded to S3 and can be checked / confirmed) and plans successfully for each workspace / project.

However, when an atlantis apply is run the following error occurs:

building command for dir "<project_name>": cannot specify a project name unless an atlantis.yaml file exists to configure projects

This error immediately appears and no more applies are run (due to parallel_apply being disabled)

When investigating on the pod (we run in Kubernetes), it appears the atlantis.yaml file is only generated for the default workspace and is never copied to the other workspace folders.

The assumption is:

• an atlantis plan is initially executed from the default workspace and so Atlantis uses the generated atlantis.yaml file there to create the workspace folders and pull the contents from git.
• an atlantis apply is executed directly from the workspace folder in question and therefore the atlantis.yaml file does not exist and it cannot figure out what it is meant to apply.

This comment from last year is explaining the
same / a similar issue with how pre-workflow hooks are executed, but with a different context.

Is there a way to force Atlantis to copy the generated file into each workspace repo so applies can be successfully executed?
Are there docs on how this could be done?

The generated file cannot be downloaded from remote storage (e.g S3) during an apply as the atlantis.yaml does not exist - should it be set as a stage either before or after the plan has occurred to ensure it exists? (documentation updated needed, this would require downloading files on each plan costing $$)

Reproduction Steps

This could be emulated by downloading the file from remote storage, without having to do the autogeneration e.g:

• Have a repo with several different terraform workspaces
• Store a valid atlantis.yaml in a remote location e.g S3
• Pull this file during a pre-workflow hook
• Run plans
• Attempt to run applies
• Check the file system, the atlantis.yaml should only be present in the default workspace

Environment details

Kubernetes version: v1.21

Running a statefulset with a lot of environment variables

Atlantis version: latest / v0.19.2 / v0.17.x

Repo Config:

repos:

• id: /.*/
  apply_requirements: [approved, mergeable]
  allowed_overrides: [apply_requirements, workflow, delete_source_branch_on_merge]
  allow_custom_workflows: true

Additional Context

The script correctly generates the Atlantis config and then plans successfully.
File system is correct.
Swapping to a repo side Atlantis file with the same configuration works correctly, so it is to do with the sharing of the file across workspaces

[jamengual]

is this still happening in v0.19.8?

[snorlaX-sleeps]

Unsure @jamengual - we are still using the nightly build created by this PR: #2180
So haven't got any new features.
We require parallel plans to support workspaces with the same name aka <product>-<region_code> and need that change in before we can upgrade.

In this case, the solution would be having the atlantis.yaml added to each workspace directory, due to how applies are different to plans, but I haven't seen anything in the release notes that would indicate this has changed

[evanstachowiak]

@jamengual i can confirm I am still unable to do an atlantis apply with a custom workflow without specifying the directory and workspace.
I am getting this error: {"level":"info","ts":"2022-08-30T15:05:19.383Z","caller":"events/apply_command_runner.go:131","msg":"determined there was no project to run apply in.","

If I run with atlantis apply -d $dir -w $workspace, it works correctly.

[jamengual]

maybe this explains a bit the reason behind that #2150

[evanstachowiak]

@jamengual, I do have autoplan key set for all projects. Looking closer at this ticket, I see that I am experiencing the same issue. The generated atlantis.yaml exists in the default workspace directory on the atlantis server, but other workspace directories do not have atlantis.yaml.

[jamengual]

so then Atlantis is behaving as expected. no?

[evanstachowiak]

No, it is not. The plan will run and detect the projects correctly, but apply will not. I'm not sure why plan is running correctly if atlantis.yaml only exists in the default workspace.