[Documentation] - Update Section running-policy-check-against-terraform-source-code

[ltmleo]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Overview of the Issue

Section Running-policy-check-against-terraform-source-code do not show some important configurantions like:

1. The conftest cannot exit with a status errors than 0. So flag --no-fail needs to be added to conftest command.
2. When executing a custom policy command, the flag custom_policy_check must be enabled.

So my purpose is to change

workflows:
  custom:
    policy_check:
      steps:
        - show
        - run: conftest test $SHOWFILE *.tf

to

custom_policy_check: true
workflows:
  custom:
    policy_check:
      steps:
        - show
        - run: conftest test $SHOWFILE *.tf --no-fail

Just this information would have saved me a few hours of debugging.

Reproduction Steps

When executing something like

workflows:
  custom:
    policy_check:
      steps:
        - show
        - run: conftest test $SHOWFILE *.tf

The following , not helpful, output are shown:

Policy Check Error
unable to unmarshal conftest output

To work properly I needed to add the fields shown below

custom_policy_check: true
workflows:
  custom:
    policy_check:
      steps:
        - show
        - run: conftest test $SHOWFILE *.tf --no-fail

Logs

Logs

{"level":"error","ts":"2023-12-22T16:18:26.365-0300","caller":"events/instrumented_project_command_runner.go:78","msg":"Error running policy_check operation: unable to unmarshal conftest output","json":{"repo":"redacted","pull":"42"},"stacktrace":"github.com/runatlantis/atlantis/server/events.RunAndEmitStats\n\t/home/Documents/Github/atlantis/server/events/instrumented_project_command_runner.go:78\ngithub.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandRunner).PolicyCheck\n\t/home/Documents/Github/atlantis/server/events/instrumented_project_command_runner.go:42\ngithub.com/runatlantis/atlantis/server/events.runProjectCmds\n\t/home/vivo/Documents/Github/atlantis/server/events/project_command_pool_executor.go:48\ngithub.com/runatlantis/atlantis/server/events.(*PolicyCheckCommandRunner).Run\n\t/home/vivo/Documents/Github/atlantis/server/events/policy_check_command_runner.go:65\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).run\n\t/home/Documents/Github/atlantis/server/events/plan_command_runner.go:290\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).Run\n\t/home/Documents/Github/atlantis/server/events/plan_command_runner.go:306\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand\n\t/home//Documents/Github/atlantis/server/events/command_runner.go:365"}

Environment details

If not already included, please provide the following:

• Atlantis version: main source code and v0.27.0
• Deployment method: running locally and kubernetes
• If not running the latest Atlantis version have you tried to reproduce this issue on the latest version:
• Atlantis flags:

Atlantis server-side config file:

custom_policy_check: true
workflows:
  default:
    policy_check:
      steps:
        - show
        - run: conftest test $SHOWFILE *.tf --no-fail

Repo atlantis.yaml file:

version: 3
projects:
  - name: example
    dir: example/ex1
    autoplan:
      enabled: true
    workflow: default

Additional Context

Line 164 of https://github.com/runatlantis/atlantis/edit/main/runatlantis.io/docs/policy-checking.md

[albertorm95]

This is correct, I think except the --no-fail