Skip to content

DAG masking omits Terraform inputs for post-hooks #2906

Description

@zack-is-cool

Summary

With masking enabled, Atmos DAG execution removes selected values from the generated Terraform JSON var-file, apparently to use the documented secret-safe TF_VAR_ delivery path instead.

During post-hook execution, the corresponding value is not available to the child Terraform/OpenTofu process. The required input is absent from the var-file, causing Terraform/OpenTofu to report it as unset or to parse a malformed partial value.

The same DAG succeeds with --mask=false, which keeps the required values in the generated .tfvars.json file.

Expected behavior

Masking must not alter the effective Terraform/OpenTofu input received by a component or post-hook.

When masking omits a value from a generated var-file, Atmos must deliver the equivalent value to every child Terraform/OpenTofu process that requires it, including processes invoked through DAG execution and post-hooks.

Atmos documents masking as output interception and documents secret-safe TF_VAR_ delivery for automatically registered sensitive Terraform/OpenTofu outputs:

A direct component apply and an equivalent DAG apply should receive identical effective Terraform/OpenTofu inputs regardless of whether output masking is enabled.

Actual behavior

With masking enabled, DAG execution produces an incomplete Terraform JSON var-file for a post-hook.

Comparing the masking-enabled file with the --mask=false file shows that required top-level fields are absent when masking is enabled, including application identity, deployment metadata, tags, hook-specific configuration, a secret-name prefix, and a required multiline payload.

The omitted multiline payload explains the Terraform/OpenTofu error that the variable is not set. In other cases, partially delivered multiline values are parsed as HCL and fail on YAML syntax.

Terraform/OpenTofu reports errors such as:

Error: Extra characters after expression
Error: Unsupported operator
Error: No value for required variable

The affected hook consumes non-sensitive configuration. A separate sensitive output from the same component is not directly referenced by the hook, but appears to trigger the masking behavior.

The same component succeeds when:

  1. Applied directly.
  2. Applied through the DAG with --mask=false.

The documented settings.terminal.mask.enabled: false setting did not fully disable this behavior during DAG execution. Only the --mask=false CLI flag allowed the DAG to complete.

Steps to reproduce

  1. Configure a Terraform component with:
    • A non-sensitive multiline string value consumed by a post-hook.
    • A separate sensitive output.
  2. Configure an Atmos post-hook that consumes only the non-sensitive value.
  3. Include the component and hook in an Atmos dependency graph.
  4. Run the component directly with masking enabled.
    • Expected and observed: succeeds.
  5. Run the dependency graph with masking enabled.
    • Observed: the generated Terraform JSON var-file is missing required fields and the post-hook fails.
  6. Run the same DAG command with --mask=false.
    • Expected and observed: succeeds.
  7. Compare the generated JSON var-files from both runs.
    • Observed: required top-level fields are absent from the masking-enabled file.

Multiline structured values

The failing value is a large multiline structured string. This may be an important trigger: if it is delivered through a Terraform/OpenTofu CLI-variable path rather than a JSON var-file, it can be parsed as HCL instead of preserved as a string.

The masking-enabled DAG/post-hook path may therefore have two related failures:

  1. A value omitted from the JSON var-file is not propagated to the child process through TF_VAR_.
  2. When a partial or fallback CLI-variable path is used, multiline structured content is parsed as HCL and fails.

A short scalar value may present only as an unset variable, while a multiline structured value makes the corruption visible through parser errors.

Impact

This is not only a log-redaction issue. It can omit or invalidate Terraform/OpenTofu inputs used by post-hooks, preventing otherwise valid components from deploying through Atmos DAG execution.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions