Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions website/docs/functions/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -92,11 +92,11 @@ If you must use template functions:

If you have existing template functions, consider migrating to YAML functions:

| Instead of (Template) | Use (YAML) |
|----------------------|------------|
| `{{ exec "command" }}` | `!exec command` |
| `{{ env "VAR" }}` | `!env VAR` |
| `{{ toJson .values }}` | Native YAML structures |
| Instead of (Template) | Use (YAML) |
|-------------------------------|-------------------------------------------|
| `{{ exec "command" }}` | `!exec command` |
| `{{ env "VAR" }}` | `!env VAR` |
| `{{ toJson .values }}` | Native YAML structures |
| Reading outputs via templates | `!terraform.output` or `!terraform.state` |

## Performance Considerations
Expand Down
43 changes: 43 additions & 0 deletions website/docs/functions/yaml/env.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,49 @@ The `env:` section follows the standard Atmos merge priority:
This means values defined at the component level will override values from globals.
:::

## Deferred Evaluation During Merge

Atmos uses **deferred evaluation** when processing YAML functions during configuration merging. This prevents type conflicts that can occur when inheriting and overriding values across multiple stack layers.

### The Problem: Type Conflicts

Without deferred evaluation, mixing concrete values and YAML functions in the inheritance chain would cause errors:

```yaml
# Base catalog (catalog/app/defaults.yaml)
components:
terraform:
my-app:
vars:
database_url: "postgresql://localhost:5432/mydb" # Concrete string value

# Environment override (stacks/prod/app.yaml)
components:
terraform:
my-app:
vars:
database_url: !env DATABASE_URL # YAML function - different type!
```

In this scenario, Atmos would attempt to merge a string (`"postgresql://localhost:5432/mydb"`) with a YAML function reference during the configuration merge process, resulting in a type mismatch.

### The Solution: Deferred Merge

Atmos now **defers the evaluation** of YAML functions (including `!env`, `!terraform.output`, `!terraform.state`, `!template`, `!store`, `!store.get`, and `!exec`) until after all configuration layers have been merged. This three-phase approach eliminates type conflicts:

1. **Defer Phase** - YAML functions are identified and temporarily replaced with placeholders
2. **Merge Phase** - All configuration layers merge without type conflicts
3. **Evaluate Phase** - YAML functions are evaluated and applied to the final merged result

With deferred evaluation, the example above works seamlessly. The YAML function in the override layer is deferred during merge, then evaluated after merging completes, correctly replacing the base value.

### Benefits

- **Flexible Configuration Patterns** - Mix static values and YAML functions across inheritance layers without conflicts
- **Gradual Migration** - Migrate from static to dynamic configurations incrementally
- **Team Collaboration** - Different teams can use different approaches (static vs. templated) in their layers
- **Multi-Environment Support** - Use static values in dev/staging and YAML functions in production

## Limitations

**Single-Pass Processing**: YAML functions are processed in a single pass. This means `!env` cannot reference environment variables that are defined by other YAML functions in the same component section.
Expand Down
43 changes: 43 additions & 0 deletions website/docs/functions/yaml/exec.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -67,3 +67,46 @@ You can use [Atmos Stack Manifest Templating](/core-concepts/stacks/templates) i
Atmos processes the templates first, and then executes the `!exec` function, allowing you to provide the parameters to
the function dynamically.
:::

## Deferred Evaluation During Merge

Atmos uses **deferred evaluation** when processing YAML functions during configuration merging. This prevents type conflicts that can occur when inheriting and overriding values across multiple stack layers.

### The Problem: Type Conflicts

Without deferred evaluation, mixing concrete values and YAML functions in the inheritance chain would cause errors:

```yaml
# Base catalog (catalog/app/defaults.yaml)
components:
terraform:
my-app:
vars:
build_number: "1.0.0" # Concrete string value

# Environment override (stacks/prod/app.yaml)
components:
terraform:
my-app:
vars:
build_number: !exec cat VERSION # YAML function - different type!
```

In this scenario, Atmos would attempt to merge a string (`"1.0.0"`) with a YAML function reference during the configuration merge process, resulting in a type mismatch.

### The Solution: Deferred Merge

Atmos now **defers the evaluation** of YAML functions (including `!exec`, `!terraform.output`, `!terraform.state`, `!template`, `!store`, `!store.get`, and `!env`) until after all configuration layers have been merged. This three-phase approach eliminates type conflicts:

1. **Defer Phase** - YAML functions are identified and temporarily replaced with placeholders
2. **Merge Phase** - All configuration layers merge without type conflicts
3. **Evaluate Phase** - YAML functions are evaluated and applied to the final merged result

With deferred evaluation, the example above works seamlessly. The YAML function in the override layer is deferred during merge, then evaluated after merging completes, correctly replacing the base value.

### Benefits

- **Flexible Configuration Patterns** - Mix static values and YAML functions across inheritance layers without conflicts
- **Gradual Migration** - Migrate from static to dynamic configurations incrementally
- **Team Collaboration** - Different teams can use different approaches (static vs. templated) in their layers
- **Multi-Environment Support** - Use static values in dev/staging and YAML functions in production
49 changes: 46 additions & 3 deletions website/docs/functions/yaml/store.get.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -78,20 +78,63 @@ If you need to retrieve values that follow the Atmos stack/component/key pattern
[`!store`](/functions/yaml/store) function instead, as it provides better integration with Atmos component dependencies.
:::

## Deferred Evaluation During Merge

Atmos uses **deferred evaluation** when processing YAML functions during configuration merging. This prevents type conflicts that can occur when inheriting and overriding values across multiple stack layers.

### The Problem: Type Conflicts

Without deferred evaluation, mixing concrete values and YAML functions in the inheritance chain would cause errors:

```yaml
# Base catalog (catalog/app/defaults.yaml)
components:
terraform:
my-app:
vars:
api_key: "default-key-12345" # Concrete string value

# Environment override (stacks/prod/app.yaml)
components:
terraform:
my-app:
vars:
api_key: !store.get ssm /external/api-key # YAML function - different type!
```

In this scenario, Atmos would attempt to merge a string (`"default-key-12345"`) with a YAML function reference during the configuration merge process, resulting in a type mismatch.

### The Solution: Deferred Merge

Atmos now **defers the evaluation** of YAML functions (including `!store.get`, `!store`, `!terraform.output`, `!terraform.state`, `!template`, `!exec`, and `!env`) until after all configuration layers have been merged. This three-phase approach eliminates type conflicts:

1. **Defer Phase** - YAML functions are identified and temporarily replaced with placeholders
2. **Merge Phase** - All configuration layers merge without type conflicts
3. **Evaluate Phase** - YAML functions are evaluated and applied to the final merged result

With deferred evaluation, the example above works seamlessly. The YAML function in the override layer is deferred during merge, then evaluated after merging completes, correctly replacing the base value.

### Benefits

- **Flexible Configuration Patterns** - Mix static values and YAML functions across inheritance layers without conflicts
- **Gradual Migration** - Migrate from static to dynamic configurations incrementally
- **Team Collaboration** - Different teams can use different approaches (static vs. templated) in their layers
- **Multi-Environment Support** - Use static values in dev/staging and YAML functions in production

### Using YQ Expressions

You can use [YQ](https://mikefarah.gitbook.io/yq) expressions to extract specific values
from complex data structures:

<File title="stack.yaml">
```yaml
# Extract a field from a JSON object
# Extract database host from app-config in Redis
database_host: !store.get redis app-config | query .database.host

# Get the first item from an array
# Get the first availability zone from Azure Key Vault array
primary_zone: !store.get azure-keyvault availability-zones | query .[0]

# Extract nested values with a default
# Extract API key from SSM config with empty object fallback
api_key: !store.get ssm /external/config | default "{}" | query .api.key
```
</File>
Expand Down
43 changes: 43 additions & 0 deletions website/docs/functions/yaml/store.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,49 @@ Atmos processes the templates first, and then executes the `!store` function, al
the function dynamically.
:::

## Deferred Evaluation During Merge

Atmos uses **deferred evaluation** when processing YAML functions during configuration merging. This prevents type conflicts that can occur when inheriting and overriding values across multiple stack layers.

### The Problem: Type Conflicts

Without deferred evaluation, mixing concrete values and YAML functions in the inheritance chain would cause errors:

```yaml
# Base catalog (catalog/vpc/defaults.yaml)
components:
terraform:
vpc:
vars:
subnet_ids: ["subnet-111", "subnet-222"] # Concrete list value

# Environment override (stacks/prod/networking.yaml)
components:
terraform:
vpc:
vars:
subnet_ids: !store redis networking vpc subnet_ids # YAML function - different type!
```

In this scenario, Atmos would attempt to merge a list (`["subnet-111", "subnet-222"]`) with a YAML function reference during the configuration merge process, resulting in a type mismatch.

### The Solution: Deferred Merge

Atmos now **defers the evaluation** of YAML functions (including `!store`, `!store.get`, `!terraform.output`, `!terraform.state`, `!template`, `!exec`, and `!env`) until after all configuration layers have been merged. This three-phase approach eliminates type conflicts:

1. **Defer Phase** - YAML functions are identified and temporarily replaced with placeholders
2. **Merge Phase** - All configuration layers merge without type conflicts
3. **Evaluate Phase** - YAML functions are evaluated and applied to the final merged result

With deferred evaluation, the example above works seamlessly. The YAML function in the override layer is deferred during merge, then evaluated after merging completes, correctly replacing the base value.

### Benefits

- **Flexible Configuration Patterns** - Mix static values and YAML functions across inheritance layers without conflicts
- **Gradual Migration** - Migrate from static to dynamic configurations incrementally
- **Team Collaboration** - Different teams can use different approaches (static vs. templated) in their layers
- **Multi-Environment Support** - Use static values in dev/staging and YAML functions in production

## Using YQ Expressions to retrieve individual values

To retrieve individual values from complex types such as maps and lists, or do any kind of filtering or querying,
Expand Down
43 changes: 43 additions & 0 deletions website/docs/functions/yaml/template.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,49 @@ YAML function. It produces the same results, correctly handles the complex types

:::

## Deferred Evaluation During Merge

Atmos uses **deferred evaluation** when processing YAML functions during configuration merging. This prevents type conflicts that can occur when inheriting and overriding values across multiple stack layers.

### The Problem: Type Conflicts

Without deferred evaluation, mixing concrete values and YAML functions in the inheritance chain would cause errors:

```yaml
# Base catalog (catalog/vpc/defaults.yaml)
components:
terraform:
vpc:
vars:
security_group_ids: ["sg-111", "sg-222"] # Concrete list value

# Environment override (stacks/prod/networking.yaml)
components:
terraform:
vpc:
vars:
security_group_ids: !template '{{ toJson (atmos.Component "security" .stack).outputs.group_ids }}' # YAML function - different type!
```

In this scenario, Atmos would attempt to merge a list (`["sg-111", "sg-222"]`) with a YAML function reference during the configuration merge process, resulting in a type mismatch.

### The Solution: Deferred Merge

Atmos now **defers the evaluation** of YAML functions (including `!template`, `!terraform.output`, `!terraform.state`, `!store`, `!store.get`, `!exec`, and `!env`) until after all configuration layers have been merged. This three-phase approach eliminates type conflicts:

1. **Defer Phase** - YAML functions are identified and temporarily replaced with placeholders
2. **Merge Phase** - All configuration layers merge without type conflicts
3. **Evaluate Phase** - YAML functions are evaluated and applied to the final merged result

With deferred evaluation, the example above works seamlessly. The YAML function in the override layer is deferred during merge, then evaluated after merging completes, correctly replacing the base value.

### Benefits

- **Flexible Configuration Patterns** - Mix static values and YAML functions across inheritance layers without conflicts
- **Gradual Migration** - Migrate from static to dynamic configurations incrementally
- **Team Collaboration** - Different teams can use different approaches (static vs. templated) in their layers
- **Multi-Environment Support** - Use static values in dev/staging and YAML functions in production

## Advanced Examples

The `!template` Atmos YAML function can be used to make your stack configuration DRY and reusable.
Expand Down
46 changes: 46 additions & 0 deletions website/docs/functions/yaml/terraform.output.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,49 @@ Atmos processes the templates first, and then executes the `!terraform.output` f
the function dynamically.
:::

## Deferred Evaluation During Merge

Atmos uses **deferred evaluation** when processing YAML functions during configuration merging. This prevents type conflicts that can occur when inheriting and overriding values across multiple stack layers.

### The Problem: Type Conflicts

Without deferred evaluation, mixing concrete values and YAML functions in the inheritance chain would cause errors:

```yaml
# Base catalog (catalog/vpc/defaults.yaml)
components:
terraform:
vpc:
vars:
cidr_block: "10.0.0.0/16" # Concrete string value

# Environment override (stacks/prod/networking.yaml)
components:
terraform:
vpc:
vars:
cidr_block: !terraform.output network-config cidr_block # YAML function - different type!
```

In this scenario, Atmos would attempt to merge a string (`"10.0.0.0/16"`) with a YAML function reference during the configuration merge process, resulting in a type mismatch.

### The Solution: Deferred Merge

Atmos now **defers the evaluation** of YAML functions (including `!terraform.output`, `!terraform.state`, `!template`, `!store.get`, `!exec`, and `!env`) until after all configuration layers have been merged. This three-phase approach eliminates type conflicts:

1. **Defer Phase** - YAML functions are identified and temporarily replaced with placeholders
2. **Merge Phase** - All configuration layers merge without type conflicts
3. **Evaluate Phase** - YAML functions are evaluated and applied to the final merged result

With deferred evaluation, the example above works seamlessly. The YAML function in the override layer is deferred during merge, then evaluated after merging completes, correctly replacing the base value.

### Benefits

- **Flexible Configuration Patterns** - Mix static values and YAML functions across inheritance layers without conflicts
- **Gradual Migration** - Migrate from static to dynamic configurations incrementally
- **Team Collaboration** - Different teams can use different approaches (static vs. templated) in their layers
- **Multi-Environment Support** - Use static values in dev/staging and YAML functions in production

## `!terraform.output` Function Execution Flow

When processing the `!terraform.output` YAML function for a component in a stack, Atmos executes the following steps:
Expand Down Expand Up @@ -147,6 +190,9 @@ endpoint: !terraform.output services '.endpoints["my-service"]["production"].url

# Combine with stack templating
token: !terraform.output identity {{ .stack }} '.tokens["github-actions"].value'

# Escape single quotes by doubling them when needed inside the expression
app_name: !terraform.output config '.apps["app''s-name"].display_name'
```

**Quote escaping rules**
Expand Down
Loading