Skip to content

fix: Resolve multiple regressions and reported issues#2035

Closed
aknysh wants to merge 45 commits intomainfrom
aknysh/fix-issues-5
Closed

fix: Resolve multiple regressions and reported issues#2035
aknysh wants to merge 45 commits intomainfrom
aknysh/fix-issues-5

Conversation

@aknysh
Copy link
Copy Markdown
Member

@aknysh aknysh commented Jan 29, 2026

what

  • Fix bidirectional template references between locals and settings (Atmos 1.205.0 regression)
  • Fix premature template processing during imports causing {{ .atmos_component }} failures
  • Fix spacelift_stack and atlantis_project template resolution order
  • Fix stack-level vs component-level locals separation in output
  • Fix terraform shell command routing when invoked from Atmos interactive UI
  • Fix JIT vendoring support for generate varfile and generate backend
  • Fix YAML string values ending with colons parsed incorrectly by yq expressions
  • Fix AuthManager propagation to YAML functions in terraform shell command
  • Fix Helm plugin install in Dockerfile with --verify=false
  • Fix dangling .terraform symlinks breaking describe-affected tests
  • Refactor: extract shared auth manager creation into utils_auth.go
  • Refactor: route terraform shell directly from UI dispatcher without touching ExecuteTerraform
  • Fix template settings defaulting to false when config file doesn't explicitly configure templates
  • Add blog post for templating enabled by default

why

1. Settings can't refer to locals anymore — Atmos 1.205.0 regression (#2032)

After PR #1994, locals could reference settings but settings could no longer reference locals. The root cause was that extractAndAddLocalsToContext() added raw settings/vars/env to the context without processing their templates first. Go templates don't recursively expand template strings in data values, so {{ .locals.stage }} inside a settings value remained as a literal string.

Fix: Introduced a processing pipeline in extractAndAddLocalsToContext():

  1. Resolve locals first (can reference raw settings/vars/env)
  2. Process templates in settings using resolved locals context
  3. Process templates in vars using resolved locals + processed settings
  4. Process templates in env using all of the above

Added processTemplatesInSection() helper that converts a section to YAML, processes templates, and parses the result back.

Files: internal/exec/stack_processor_utils.go, internal/exec/stack_processor_utils_test.go

2. Template regression with {{ .atmos_component }} in non-.tmpl files (#2032)

PR #1994 inadvertently caused template processing to trigger during imports for non-.tmpl files that had settings, vars, or env sections. The locals feature populated the template context, making len(context) > 0, which triggered the template processing guard. Templates like {{ .atmos_component }} then failed because component context isn't available at import time.

Fix: Track whether context was externally provided vs extracted from the file itself using originalContextProvided. Only externally-provided context triggers template processing in non-.tmpl files. Also persist resolved sections back into stackConfigMap, remove locals from import configs before merging (file-scoped isolation), and pass resolved locals through ProcessStackConfig.

Files: internal/exec/stack_processor_utils.go, internal/exec/stack_processor_process_stacks.go, internal/exec/stack_processor_utils_test.go

3. Stack-level vs component-level locals handling (#2032)

Stack-level locals (for template resolution) were incorrectly appearing in final component output, while component-level locals (user-defined) were being lost. Additionally, ProcessComponentConfig is called multiple times and map mutations polluted the global stacksMap cache.

Fix:

  • Track original component-level local keys via OriginalComponentLocals on ConfigAndStacksInfo
  • Make shallow copies of componentSection before modification to prevent cache pollution
  • Separate key tracking (first pass only) from locals merging (every pass)
  • Filter locals in postProcessTemplatesAndYamlFunctions to keep only component-level keys
  • Add locals extraction and merging in ExecuteDescribeStacks() for terraform, helmfile, and packer

Files: internal/exec/utils.go, internal/exec/describe_stacks.go, pkg/schema/schema.go, tests/cli_locals_test.go

4. spacelift_stack and atlantis_project template resolution (#2032)

Templates referencing {{ .spacelift_stack }} and {{ .atlantis_project }} returned <no value> because these values were computed after template processing.

Fix: Moved computation of spacelift_stack and atlantis_project before ProcessTmplWithDatasources in ProcessStacks().

Files: internal/exec/utils.go

5. terraform shell not working from Atmos UI (#2017)

The interactive UI dispatches commands through ExecuteTerraform(), which had no handler for the "shell" subcommand. Since terraform shell is an Atmos-only command (not a native terraform subcommand), it would fall through and attempt to execute terraform shell as a native command, which doesn't exist.

Fix: Route terraform shell directly to ExecuteTerraformShell() from the UI dispatcher in atmos.go, bypassing ExecuteTerraform() entirely. This keeps terraform.go free of shell-specific logic. Both entry paths now call ExecuteTerraformShell directly:

  • CLI path: cmd/terraform/shell.goExecuteTerraformShell()
  • UI path: atmos.goshellOptionsForUI()ExecuteTerraformShell()

Extracted testable helpers: shellInfoFromOptions() (builds ConfigAndStacksInfo from ShellOptions), resolveWorkdirPath() (returns workdir override or original path), and shellOptionsForUI() (builds ShellOptions for the UI dispatch).

Files: internal/exec/atmos.go, internal/exec/terraform_shell.go, internal/exec/terraform_shell_test.go

6. JIT vendoring for generate varfile and generate backend (#2019)

These commands used ProcessStacks() directly without triggering JIT provisioning hooks. Also, writeBackendConfigFile hardcoded path construction instead of using constructTerraformComponentWorkingDir().

Fix: Added JIT provisioning support following the ExecuteTerraform() pattern: check component path, detect source config, run auto-provisioning, use workdir path.

Files: internal/exec/terraform_generate_varfile.go, internal/exec/terraform_generate_backend.go, internal/exec/terraform_generate_varfile_test.go, internal/exec/path_utils_test.go

7. YAML strings ending with colons parsed incorrectly (#2031)

EvaluateYqExpression returned unquoted strings that yaml.Unmarshal misinterpreted as map keys when they ended with colons (e.g., AWS ARNs like arn:aws:...:secret:password::).

Fix: Added isScalarString (pre-check) and isMisinterpretedScalar (post-check) helpers to detect and return such strings directly without YAML parsing.

Files: pkg/utils/yq_utils.go, pkg/utils/yq_utils_test.go

8. AuthManager propagation to YAML functions in terraform shell

!terraform.state YAML function failed in terraform shell and terraform plan --all because AuthManager wasn't stored on configAndStacksInfo, and ExecuteTerraformShell passed nil as the authManager.

Fix: Store AuthManager on configAndStacksInfo in ProcessComponentConfig. Add Identity field to ShellOptions and --identity flag to the shell command. Extract shared auth manager creation into utils_auth.go with createAndAuthenticateAuthManager(), getMergedAuthConfig(), and storeAutoDetectedIdentity() — eliminating duplication between terraform.go and terraform_shell.go.

Files: internal/exec/utils.go, internal/exec/utils_auth.go, internal/exec/utils_auth_test.go, internal/exec/terraform_shell.go, pkg/terraform/options.go, cmd/terraform/shell.go

9. Helm plugin install in Dockerfile

Added --verify=false to helm plugin install command for Helm 4 compatibility.

Files: Dockerfile

10. Dangling .terraform symlinks in describe-affected tests

TestDescribeAffectedWith* tests failed locally due to a dangling symlink in examples/secrets-masking/.terraform/providers/ left by a previous test run.

Fix: Added .terraform to the copy skip filter alongside the existing node_modules skip.

11. Template settings defaulting to false when config file doesn't explicitly configure templates

After PR #1941 (config isolation for --chdir), fixtures with their own atmos.yaml that didn't specify template settings got templates.settings.enabled = false because Viper zero-valued bools. Previously, the git root's atmos.yaml (which had templates.settings.enabled: true) was merged in.

Fix: Added Viper defaults for templates.settings.enabled, templates.settings.sprig.enabled, and templates.settings.gomplate.enabled in setDefaultConfiguration().

Files: pkg/config/load.go

references

Summary by CodeRabbit

  • New Features

    • Templates (Sprig, Gomplate) enabled by default in Atmos.
    • Terraform shell interactive mode in UI with identity support.
  • Bug Fixes

    • Fixed bidirectional template references across locals, settings, vars, and env sections.
    • Fixed YAML parsing for strings ending with colons (e.g., ARNs).
    • Fixed template processing regression for non-.tmpl Atmos component files.
    • Added JIT vendoring support for generate backend and varfile commands.
    • Fixed Terraform state/output YAML functions authentication.
  • Chores

    • Updated multiple dependencies (storage, AWS SDK, OPA, Redis, Goldmark, and others).

aknysh and others added 15 commits January 28, 2026 11:35
…tmpl files

Fix regression in 1.205 where templates using {{ .atmos_component }} or
{{ .atmos_stack }} fail in non-.tmpl files that have a locals section.

The locals feature (commit 6ae0a27) inadvertently triggered template
processing during import by adding settings/vars/env to context, making
len(context) > 0. This caused ProcessTmpl to fail on templates that
reference component-specific variables not available during import.

The fix tracks whether context was originally provided (from outside) vs
populated from file extraction. Templates are now only processed during
import when:
1. The file has a .tmpl extension, OR
2. Context was explicitly passed from outside

Adds test case and documentation for the regression.

Co-Authored-By: Claude <noreply@anthropic.com>
Helm now requires verification by default for plugins. The helm-diff
plugin source does not support verification, so we skip it with
--verify=false flag.

Co-Authored-By: Claude <noreply@anthropic.com>
Two issues were fixed:

1. Missing AuthManager propagation in ProcessComponentConfig: The function
   only set AuthContext from the authManager, but did not store the
   AuthManager itself on configAndStacksInfo. This meant YAML functions
   like !terraform.state could not access the AuthManager for authentication.

2. Missing authentication setup in terraform shell: The ExecuteTerraformShell
   function was calling ProcessStacks with nil as the authManager parameter.
   This meant no authentication context was available when processing YAML
   functions during the shell command.

The fix:
- Set configAndStacksInfo.AuthManager = authManager in ProcessComponentConfig
- Add Identity field to ShellOptions struct
- Add --identity flag support to shell command
- Create and authenticate AuthManager in ExecuteTerraformShell before ProcessStacks

This enables YAML functions like !terraform.state to use authenticated
credentials when running atmos terraform shell.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When yq returns a scalar value ending with colons (like AWS Secrets Manager
ARNs), the YAML parser was misinterpreting them as map keys with null values.
For example, "arn:aws:...:password::" became {"password:": null}.

This fix adds helper functions to detect and handle this edge case:
- isScalarString: pre-checks if the result should be returned as-is
- isMisinterpretedScalar: post-checks if YAML parsing created a false map
- isYAMLNullValue: checks if a YAML node represents null
- keyMatchesOriginalWithColon: checks if key matches original with trailing colon

Fixes #2031

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The generate varfile and generate backend commands now properly support
JIT (Just-in-Time) vendored components. Previously, these commands failed
when a component was configured with a source attribute because they
skipped the JIT provisioning hooks.

Changes:
- Added ensureTerraformComponentExists helper for JIT provisioning check
- Added tryJITProvision helper to provision component sources
- Both commands now use these helpers to support JIT vendored components
- Fixed writeBackendConfigFile to use constructTerraformComponentWorkingDir
- Added tests for JIT vendored component path handling

Fixes #2019

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Issue: #2018

The terraform two-word commands (providers lock, state list, workspace select)
were not recognized when passed as a quoted single argument. This adds proper
parsing for both quoted and separate argument forms.

Changes:
- Add helper functions for modular two-word command parsing
- parseTwoWordCommand, parseQuotedTwoWordCommand, parseSeparateTwoWordCommand
- processTerraformTwoWordCommand, processSingleCommand
- Support all terraform two-word commands: providers, state, workspace, write
- Add comprehensive unit tests for all helper functions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Issue: #2017

When "terraform shell" was selected from the Atmos interactive UI,
it was being passed to the terraform executable which doesn't have
a native "shell" command. This adds early interception of the "shell"
subcommand in ExecuteTerraform() to route it to ExecuteTerraformShell().

Changes:
- Add "shell" subcommand handling in ExecuteTerraform()
- Convert ConfigAndStacksInfo to ShellOptions for shell execution
- Add unit tests for shell options conversion
- Add documentation for the fix

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Issue: #2032

After PR #1994, settings could no longer reference locals (regression from
1.204). This adds template processing for settings, vars, and env sections
after locals are resolved, enabling bidirectional references between all
sections.

The fix ensures:
- Locals can reference settings (resolved during locals processing)
- Settings can reference locals (new template processing step)
- Vars can reference both locals and processed settings
- Env can reference locals, processed settings, and processed vars

Added processTemplatesInSection() helper and comprehensive tests.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add comprehensive unit tests for new functions introduced in this branch:

- TestProcessTemplatesInSection: Tests template processing in sections
- TestProcessTemplatesInSection_EdgeCases: Tests edge cases like nil, empty, lists
- TestStoreAuthenticatedIdentity: Tests auth identity storage with mock AuthManager
- TestShellOptionsValidation: Tests ShellOptions field validation
- TestShellConfigWithWorkdirProvisioner: Tests workdir provisioner path handling
- TestEnsureTerraformComponentExists: Tests component existence validation
- TestTryJITProvision: Tests JIT provisioning behavior
- TestVarfileOptions_Validation: Tests VarfileOptions field validation
- TestVarfileOptions_ProcessingOptions: Tests ProcessingOptions embedding

Coverage improvements:
- processTemplatesInSection: 66.7% -> 80.0%
- ensureTerraformComponentExists: 0% -> 80.0%
- tryJITProvision: 0% -> 28.6%
- storeAuthenticatedIdentity: 0% -> 100%

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use filepath.Join() for platform-agnostic path construction in tests:
- Replace hardcoded "/tmp/test.tfvars.json" with filepath.Join()
- Replace hardcoded Unix paths in TestShellConfigWithWorkdirProvisioner

This ensures tests pass on Windows where path separator is backslash.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Move locals merging outside the OriginalComponentLocals guard to ensure
  stack-level locals are available for template processing on every call
  to ProcessComponentConfig
- Move spacelift_stack and atlantis_project computation before template
  processing so they can be referenced in templates
- Update documentation with details of both fixes

Fixes templates like {{ .locals.namespace }} and {{ .spacelift_stack }}
returning <no value> in describe component output.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@aknysh aknysh added the patch A minor, backward compatible change label Jan 29, 2026
@aknysh aknysh requested review from a team as code owners January 29, 2026 01:55
@github-actions github-actions bot added the size/xl Extra large size PR label Jan 29, 2026
@mergify
Copy link
Copy Markdown

mergify bot commented Jan 29, 2026

Warning

This PR exceeds the recommended limit of 1,000 lines.

Large PRs are difficult to review and may be rejected due to their size.

Please verify that this PR does not address multiple issues.
Consider refactoring it into smaller, more focused PRs to facilitate a smoother review process.

@mergify
Copy link
Copy Markdown

mergify bot commented Jan 29, 2026

Important

Cloud Posse Engineering Team Review Required

This pull request modifies files that require Cloud Posse's review. Please be patient, and a core maintainer will review your changes.

To expedite this process, reach out to us on Slack in the #pr-reviews channel.

@mergify mergify bot added the needs-cloudposse Needs Cloud Posse assistance label Jan 29, 2026
@github-actions
Copy link
Copy Markdown

github-actions bot commented Jan 29, 2026

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

  • go.mod

@aknysh aknysh self-assigned this Jan 29, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Jan 29, 2026

📝 Walkthrough

Walkthrough

This PR implements bidirectional template resolution between locals/settings/vars/env sections, adds Terraform shell UI support with identity-aware authentication, enables JIT vendoring for generate commands, fixes YAML parsing for strings ending with colons, and sets template processing as enabled by default.

Changes

Cohort / File(s) Summary
Core Stack Processing & Locals Resolution
internal/exec/stack_processor_utils.go, internal/exec/stack_processor_utils_test.go, internal/exec/stack_processor_process_stacks.go
Introduces multi-pass template processing with locals-first resolution; adds processTemplatesInSection helper for template-processing settings/vars/env; distinguishes externally-provided vs file-derived context to prevent premature template evaluation during imports; stores resolved sections into stack config; removes locals from imports to enforce file-scoped semantics. Extensive test coverage for bidirectional references and edge cases.
Describe Stacks & Schema Updates
internal/exec/describe_stacks.go, internal/exec/describe_stacks_test.go, pkg/schema/schema.go
Extracts and merges stack-level with component-level locals; introduces OriginalComponentLocals field in schema for tracking component-level keys; propagates merged locals through all component types (Terraform, Helmfile, Packer); adds tests for locals merging across components.
Utils & Authentication Propagation
internal/exec/utils.go, internal/exec/utils_auth.go, internal/exec/utils_auth_test.go, internal/exec/utils_test.go
Adds locals merging with precedence; implements filtering of stack-level locals from final output; stores AuthManager and propagates AuthContext for YAML functions; pre-computes spacelift_stack/atlantis_project before template processing; includes comprehensive auth orchestration and test coverage.
Terraform Shell UI Support
internal/exec/terraform_shell.go, internal/exec/terraform_shell_test.go, cmd/terraform/shell.go, pkg/terraform/options.go, internal/exec/atmos.go
Routes "shell" subcommand to new ExecuteTerraformShell handler with UI support; adds ShellOptions struct with Identity field; implements workdir resolution and auth initialization; adds shellInfoFromOptions and resolveWorkdirPath helpers; updates CLI to wire --identity flag; includes extensive test coverage.
JIT Vendoring for Generate Commands
internal/exec/terraform_generate_varfile.go, internal/exec/terraform_generate_varfile_test.go, internal/exec/terraform_generate_backend.go, internal/exec/terraform_generate_backend_test.go, internal/exec/path_utils_test.go
Adds JIT provisioning checks before generate operations; introduces helpers ensureTerraformComponentExists, tryJITProvision, checkDirectoryExists; uses constructTerraformComponentWorkingDir for backend path construction; includes comprehensive test coverage for workdir and JIT scenarios.
YAML Parsing Improvements
pkg/utils/yq_utils.go, pkg/utils/yq_utils_test.go
Fixes EvaluateYqExpression to preserve strings ending with colons; adds isScalarString, isMisinterpretedScalar, isYAMLNullValue, keyMatchesOriginalWithColon helpers to detect and prevent YAML misinterpretation of ARNs and colon-containing values; includes extensive test coverage for regression scenarios.
Terraform Core Changes
internal/exec/terraform.go
Simplifies auth merging flow by introducing createAndAuthenticateAuthManager function; stores auth result in info.AuthManager for nested operations; maintains consistent error handling for user aborts.
Configuration & Defaults
pkg/config/load.go, Dockerfile
Sets template processing defaults to enabled (templates.settings.enabled/sprig.enabled/gomplate.enabled); updates Helm plugin installation with --verify=false flag for Helm 4 compatibility.
Documentation & Fixtures
docs/fixes/*, tests/fixtures/scenarios/atmos-pro-template-regression/*
Adds comprehensive fix documentation for template regression, JIT vendoring, bidirectional references, YAML colon parsing, and shell UI support; includes test fixtures for Atmos Pro template regression scenarios.
Dependency & Test Snapshot Updates
go.mod, NOTICE, tests/snapshots/*, tests/cli_locals_test.go, internal/exec/describe_affected_test.go
Updates Go dependencies to latest versions; refreshes license NOTICE file; updates snapshots to reflect new template.settings defaults; adds test cache clearing for locals test isolation.

Sequence Diagram(s)

sequenceDiagram
    participant CLI as CLI/UI
    participant Shell as ExecuteTerraformShell
    participant Auth as AuthManager
    participant Stack as ProcessStacks
    participant YAML as YAML Functions

    CLI->>Shell: shellInfoFromOptions(component, stack, identity)
    Shell->>Auth: createAndAuthenticateAuthManager(config, info)
    Auth->>Auth: mergeGlobalAndComponentAuth()
    Auth->>Auth: authenticateWithIdentity()
    Auth->>Shell: return authManager
    Shell->>Stack: ProcessStacks(info with authManager)
    Stack->>YAML: !terraform.state/.../etc
    YAML->>Auth: use authManager.AuthContext for credentials
    YAML->>Stack: return resolved values
    Stack->>Shell: return component config
    Shell->>CLI: shell session ready
Loading
sequenceDiagram
    participant Parser as YAML Parser
    participant YQ as EvaluateYqExpression
    participant Check as Scalar Checks
    participant Result as Result Handler

    Parser->>YQ: evaluate expression
    YQ->>Check: isScalarString(result)?
    alt Scalar String
        Check->>Result: return original string
    else YAML Parse Candidate
        YQ->>YQ: unmarshal to YAML
        YQ->>Check: isMisinterpretedScalar(node, original)?
        alt Misinterpreted Colon
            Check->>Result: return original string
        else Valid YAML
            YQ->>Result: return parsed structure
        end
    end
Loading
sequenceDiagram
    participant Processor as StackProcessor
    participant Locals as extractAndAddLocalsToContext
    participant Template as processTemplatesInSection
    participant Context as Context Builder

    Processor->>Locals: extract file-scoped locals
    Locals->>Context: build localsContext
    Processor->>Template: processTemplatesInSection(settings, localsContext)
    Template->>Context: build settingsContext (locals+settings)
    Processor->>Template: processTemplatesInSection(vars, settingsContext)
    Context->>Template: build varsContext (locals+settings+vars)
    Processor->>Template: processTemplatesInSection(env, varsContext)
    Template->>Processor: return resolved sections
    Processor->>Processor: remove locals from import configs<br/>(file-scoped semantics)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested labels

minor

Suggested reviewers

  • osterman
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the primary change: resolving multiple regressions and reported issues across template processing, locals/settings/vars/env resolution, CLI handling, JIT vendoring, YAML parsing, auth propagation, and test stability.
Linked Issues check ✅ Passed The changeset implements coding solutions addressing all four linked issues: #2032 (bidirectional locals/settings references), #2017 (terraform shell UI support), #2019 (JIT vendoring for generate commands), and #2031 (YAML scalar colon parsing).
Out of Scope Changes check ✅ Passed All changes align with stated objectives: template processing improvements, locals/settings/vars/env handling, CLI shell support, JIT vendoring, YAML parsing fixes, auth propagation, Dockerfile/test updates, and Viper defaults for template settings.
Docstring Coverage ✅ Passed Docstring coverage is 87.58% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch aknysh/fix-issues-5

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@aknysh
Copy link
Copy Markdown
Member Author

aknysh commented Jan 29, 2026

@coderabbitai full review please

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Jan 29, 2026

@aknysh I'll perform a full review of all the changes in this PR.


🧠 Learnings used
Learnt from: Listener430
Repo: cloudposse/atmos PR: 934
File: tests/fixtures/scenarios/docs-generate/README.md.gotmpl:99-118
Timestamp: 2025-01-25T03:51:57.689Z
Learning: For the cloudposse/atmos repository, changes to template contents should be handled in dedicated PRs and are typically considered out of scope for PRs focused on other objectives.

Learnt from: osterman
Repo: cloudposse/atmos PR: 1686
File: docs/prd/tool-dependencies-integration.md:58-64
Timestamp: 2025-12-13T06:07:37.766Z
Learning: cloudposse/atmos: For PRD docs (docs/prd/*.md), markdownlint issues like MD040/MD010/MD034 can be handled in a separate documentation cleanup commit and should not block the current PR.

Learnt from: aknysh
Repo: cloudposse/atmos PR: 944
File: go.mod:206-206
Timestamp: 2025-01-17T00:18:57.769Z
Learning: For indirect dependencies with license compliance issues in the cloudposse/atmos repository, the team prefers to handle them in follow-up PRs rather than blocking the current changes, as these issues often require deeper investigation of the dependency tree.
✅ Actions performed

Full review triggered.

@codecov
Copy link
Copy Markdown

codecov bot commented Jan 29, 2026

Codecov Report

❌ Patch coverage is 78.80795% with 64 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.76%. Comparing base (a20e3fe) to head (359729b).
⚠️ Report is 14 commits behind head on main.

Files with missing lines Patch % Lines
internal/exec/terraform_generate_varfile.go 57.50% 9 Missing and 8 partials ⚠️
internal/exec/describe_stacks.go 71.42% 6 Missing and 6 partials ⚠️
internal/exec/terraform_shell.go 56.00% 11 Missing ⚠️
internal/exec/stack_processor_utils.go 89.47% 3 Missing and 3 partials ⚠️
internal/exec/utils_auth.go 85.00% 4 Missing and 2 partials ⚠️
internal/exec/utils.go 91.66% 2 Missing and 2 partials ⚠️
cmd/terraform/shell.go 0.00% 2 Missing ⚠️
internal/exec/atmos.go 71.42% 2 Missing ⚠️
internal/exec/terraform_generate_backend.go 60.00% 1 Missing and 1 partial ⚠️
pkg/utils/yq_utils.go 93.33% 1 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2035      +/-   ##
==========================================
+ Coverage   75.65%   75.76%   +0.10%     
==========================================
  Files         793      793              
  Lines       73892    74135     +243     
==========================================
+ Hits        55901    56165     +264     
+ Misses      14498    14469      -29     
- Partials     3493     3501       +8     
Flag Coverage Δ
unittests 75.76% <78.80%> (+0.10%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
internal/exec/stack_processor_process_stacks.go 86.24% <100.00%> (+0.07%) ⬆️
internal/exec/terraform.go 59.33% <100.00%> (+0.05%) ⬆️
pkg/config/load.go 78.78% <100.00%> (+0.10%) ⬆️
pkg/schema/schema.go 88.98% <ø> (ø)
cmd/terraform/shell.go 33.33% <0.00%> (-1.10%) ⬇️
internal/exec/atmos.go 23.85% <71.42%> (+3.26%) ⬆️
internal/exec/terraform_generate_backend.go 82.14% <60.00%> (+4.94%) ⬆️
pkg/utils/yq_utils.go 85.32% <93.33%> (+9.07%) ⬆️
internal/exec/utils.go 86.29% <91.66%> (+3.88%) ⬆️
internal/exec/stack_processor_utils.go 80.32% <89.47%> (+0.62%) ⬆️
... and 4 more

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Jan 29, 2026

📝 Walkthrough

Walkthrough

This PR introduces Terraform shell subcommand support with authentication integration, bidirectional template resolution across locals/settings/vars sections, just-in-time component provisioning, YAML scalar handling for colon-ending strings, and refactored CLI parsing for Terraform two-word commands.

Changes

Cohort / File(s) Summary
Dockerfile & Dependencies
Dockerfile, go.mod, NOTICE
Helm-diff plugin flag update, dependency version bumps (cloud.google.com/go/storage, aws-sdk-go-v2, terraform-config-inspect, opa, redis, yuin/goldmark, etc.), and license URL updates.
Terraform Shell Execution with Auth
cmd/terraform/shell.go, internal/exec/terraform_shell.go, internal/exec/terraform_shell_test.go, pkg/terraform/options.go
Adds Terraform shell subcommand support with identity/authentication propagation, including AuthManager initialization, identity wiring through ShellOptions, and helper functions for merged auth config retrieval.
CLI Two-Word Command Parsing
internal/exec/cli_utils.go, internal/exec/cli_utils_test.go
Refactors Terraform two-word command parsing (e.g., "providers lock", "state list") into modular helpers with support for both quoted and separate forms; adds parseTwoWordCommand, parseQuotedTwoWordCommand, and related parsing functions with extensive test coverage.
Stack Locals Merging & Bidirectional Templates
internal/exec/describe_stacks.go, internal/exec/stack_processor_utils.go, internal/exec/stack_processor_utils_test.go, internal/exec/utils.go, pkg/schema/schema.go
Merges stack-level and component-level locals for template processing, enables bidirectional references between settings/locals/vars, tracks original component locals for restoration, and adds comprehensive tests for bidirectional reference scenarios.
JIT Component Provisioning
internal/exec/terraform_generate_varfile.go, internal/exec/terraform_generate_varfile_test.go, internal/exec/terraform_generate_backend.go, internal/exec/path_utils_test.go
Adds just-in-time provisioning checks before varfile/backend generation, ensures Terraform component paths exist (via AutoProvisionSource), and validates workdir precedence for provisioned components.
YAML Scalar Handling
pkg/utils/yq_utils.go, pkg/utils/yq_utils_test.go
Fixes YAML parsing for strings ending with colons (e.g., ARNs) by introducing isScalarString and isMisinterpretedScalar helpers to prevent misinterpretation as maps.
Stack Describe & Output
internal/exec/describe_affected_test.go, internal/exec/stack_processor_process_stacks.go
Extends .terraform directory skipping in test fixtures, propagates resolved locals into stack describe output for template access.
Test Infrastructure & Fixtures
tests/cli_locals_test.go, tests/fixtures/scenarios/atmos-pro-template-regression/...
Updates test initialization with relaxed mode and clean state reset; adds comprehensive test scenario fixtures for atmos-pro template regression including atmos.yaml, Terraform component, and stack/mixin YAML definitions.
Documentation
docs/fixes/atmos-pro-template-regression.md, docs/fixes/generate-varfile-backend-jit-vendoring.md, docs/fixes/settings-locals-bidirectional-references.md, docs/fixes/terraform-providers-lock-command.md, docs/fixes/terraform-shell-ui-mode.md, docs/fixes/terraform-state-yaml-colon-parsing.md, docs/fixes/terraform-state-yaml-function-auth.md
Comprehensive documentation of regression fixes, new features, implementation details, reproduction steps, and test coverage for bidirectional template resolution, JIT provisioning, two-word command parsing, shell execution, YAML scalar handling, and authentication integration.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI
    participant TerraformShell as Terraform Shell<br/>Executor
    participant AuthMgr as Auth Manager
    participant Config as Component Config
    participant Shell as Interactive Shell
    
    User->>CLI: atmos terraform shell --identity=<id>
    CLI->>TerraformShell: ExecuteTerraformShell(ShellOptions{Identity})
    TerraformShell->>AuthMgr: createShellAuthManager(identity)
    AuthMgr->>Config: Fetch merged auth config<br/>(global + component-level)
    Config-->>AuthMgr: Merged auth config
    AuthMgr->>AuthMgr: Authenticate with identity
    AuthMgr-->>TerraformShell: Authenticated AuthManager
    TerraformShell->>TerraformShell: Store AuthManager in ConfigAndStacksInfo
    TerraformShell->>TerraformShell: ProcessStacks (with AuthManager)
    TerraformShell->>Shell: Launch interactive shell<br/>(with environment)
    Shell-->>User: Shell prompt
Loading
sequenceDiagram
    participant User
    participant VarfileCmd as Generate Varfile<br/>Command
    participant JIT as JIT Provisioner
    participant ComponentPath as Component Path
    participant Varfile as Varfile<br/>Generator
    
    User->>VarfileCmd: terraform generate varfile
    VarfileCmd->>VarfileCmd: ProcessStacks (resolve config)
    VarfileCmd->>VarfileCmd: ensureTerraformComponentExists()
    VarfileCmd->>ComponentPath: Check if component path exists
    ComponentPath-->>VarfileCmd: Path missing
    VarfileCmd->>JIT: tryJITProvision(component source)
    JIT->>JIT: AutoProvisionSource (5min timeout)
    JIT-->>VarfileCmd: Provisioned with workdir path
    VarfileCmd->>ComponentPath: Re-check component path
    ComponentPath-->>VarfileCmd: Path exists
    VarfileCmd->>Varfile: Generate with workdir path
    Varfile-->>User: Varfile output
Loading
sequenceDiagram
    participant Stack as Stack Processor
    participant StackLocals as Stack-Level<br/>Locals
    participant ComponentLocals as Component-Level<br/>Locals
    participant Settings as Settings Section
    participant Templates as Template<br/>Processing
    
    Stack->>StackLocals: Extract file-scoped locals
    Stack->>ComponentLocals: Extract component locals
    Stack->>ComponentLocals: Preserve original keys
    Stack->>StackLocals: Merge StackLocals + ComponentLocals<br/>(component precedence)
    Stack->>Templates: Add merged locals to context
    Stack->>Settings: Process settings templates<br/>(with merged locals)
    Stack->>Settings: Process vars templates<br/>(with resolved settings)
    Templates-->>Settings: Template resolution complete
    Stack->>Stack: Filter output to original<br/>component locals only
    Stack-->>Stack: Return ConfigAndStacksInfo<br/>(cleaned locals)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related PRs

Suggested reviewers

  • aknysh
  • osterman
🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the primary changes: resolving multiple regressions and reported issues across various Atmos components.
Docstring Coverage ✅ Passed Docstring coverage is 94.29% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch aknysh/fix-issues-5

Warning

Review ran into problems

🔥 Problems

Errors were encountered while retrieving linked issues.

Errors (1)
  • ISSUES-5: Entity not found: Issue - Could not find referenced Issue.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
NOTICE (1)

1-1655: ⚠️ Potential issue | 🔴 Critical

Regenerate the NOTICE file using the provided script.

The pipeline is failing because the NOTICE file is out of sync. Run ./scripts/generate-notice.sh locally and commit the result. Based on learnings, this file is programmatically generated and should not be manually edited.

coderabbitai[bot]
coderabbitai bot previously approved these changes Feb 1, 2026
Move the "terraform shell" dispatch from ExecuteTerraform (terraform.go)
to the UI dispatcher (atmos.go), so ExecuteTerraform no longer needs to
know about the shell subcommand. Both entry paths now call
ExecuteTerraformShell directly:
- CLI path: cmd/terraform/shell.go -> ExecuteTerraformShell
- UI path: atmos.go -> ExecuteTerraformShell

Extract testable helpers from inline logic:
- shellInfoFromOptions: builds ConfigAndStacksInfo from ShellOptions
- resolveWorkdirPath: returns workdir override or original path
- shellOptionsForUI: builds ShellOptions for the UI dispatch path

Replace tautological tests that re-implemented production logic inline
with tests that call the actual extracted functions.

Add blog post for templating enabled by default.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
coderabbitai[bot]
coderabbitai bot previously approved these changes Feb 2, 2026
…comment

Restore blog posts and documentation files that were inadvertently deleted
or reverted by commit 6e3ad09 (splitting website updates to fix-issues-6).
Add clarifying comment to utils_auth.go explaining why auth orchestration
lives in internal/exec rather than pkg/auth (circular import constraint).

Restored from main:
- website/blog/2025-01-06-chdir-config-isolation.mdx
- website/blog/2026-01-29-artifactory-store-fix.mdx
- website/docs/cli/configuration/stores.mdx
- website/docs/stacks/hooks.mdx
- website/docs/tutorials/sharing-state/stores.mdx

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

## What Changed

Atmos now registers Viper-level defaults for all three template processing settings:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this is too technical. This needs to be user-facing, not developer-facing. Users have no idea what Viper is.


## Why This Changed

The [config isolation feature](/changelog/introducing-chdir-flag) (PR #1941) changed how Atmos discovers configuration files. When you use `--chdir` or have a local `atmos.yaml`, Atmos now respects that config in isolation rather than merging it with the repository root's `atmos.yaml`.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know that this has anything to do with the stated title of this changelog entry. Maybe it belongs in a different changelog entry?


The [config isolation feature](/changelog/introducing-chdir-flag) (PR #1941) changed how Atmos discovers configuration files. When you use `--chdir` or have a local `atmos.yaml`, Atmos now respects that config in isolation rather than merging it with the repository root's `atmos.yaml`.

This created an unintended side effect: if your local `atmos.yaml` didn't explicitly set `templates.settings.enabled: true`, the value defaulted to `false` (Go's zero value for booleans). Previously, the repository root's `atmos.yaml` would have been merged in as a fallback, providing the `true` value.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be masking the underlying problem. Changing the default means the underlying problem is still there, it's just by default working under this one circumstance. I don't understand the fix. Seems like curve fitting to me.

@mergify
Copy link
Copy Markdown

mergify bot commented Feb 7, 2026

💥 This pull request now has conflicts. Could you fix it @aknysh? 🙏

@mergify mergify bot added the conflict This PR has conflicts label Feb 7, 2026
@osterman osterman closed this Feb 11, 2026
@mergify mergify bot removed conflict This PR has conflicts needs-cloudposse Needs Cloud Posse assistance labels Feb 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

patch A minor, backward compatible change size/xl Extra large size PR

Projects

None yet

2 participants