-
Notifications
You must be signed in to change notification settings - Fork 144
Description
Hi, this is an AI-drafted — forgive the robotic prose, the bug is real though:
Description
When using SubstituteWithOptions with a custom ReplacementFunc that preserves unset variables, variables without defaults are incorrectly substituted to empty strings when they appear after variables with defaults in the same string.
Environment
- Package:
github.com/compose-spec/compose-go - Affected Versions: v1.20.2, v2.10.1 (latest), and likely all versions in between
- File:
template/template.go - Function:
DefaultReplacementAppliedFunc(line ~199)
Steps to Reproduce
package main
import (
"fmt"
"strings"
"github.com/compose-spec/compose-go/template"
)
func main() {
// Mapping that returns false for all variables (none are set)
mapping := func(k string) (string, bool) {
return "", false
}
// Custom replacement function that preserves unset variables
options := []template.Option{
template.WithReplacementFunction(func(s string, m template.Mapping, cfg *template.Config) (string, error) {
result, applied, err := template.DefaultReplacementAppliedFunc(s, m, cfg)
if err == nil && !applied {
return s, nil // Keep unset variables as-is
}
return result, err
}),
template.WithoutLogging,
}
// Test: variable WITH default, then variable WITHOUT default
input := "--file ${FILE:-/tmp/file.jpg} --entry_id ${ENTRY_ID}"
result, _ := template.SubstituteWithOptions(input, mapping, options...)
fmt.Printf("Input: %q\n", input)
fmt.Printf("Expected: %q\n", "--file /tmp/file.jpg --entry_id ${ENTRY_ID}")
fmt.Printf("Actual: %q\n", result)
}Expected Behavior
Input: "--file ${FILE:-/tmp/file.jpg} --entry_id ${ENTRY_ID}"
Expected: "--file /tmp/file.jpg --entry_id ${ENTRY_ID}"
Actual: "--file /tmp/file.jpg --entry_id ${ENTRY_ID}"
The ${ENTRY_ID} variable should be preserved as-is because:
- It's not set in the mapping (returns
false) - The custom
ReplacementFuncreturns the original string when!applied
Actual Behavior
Input: "--file ${FILE:-/tmp/file.jpg} --entry_id ${ENTRY_ID}"
Expected: "--file /tmp/file.jpg --entry_id ${ENTRY_ID}"
Actual: "--file /tmp/file.jpg --entry_id "
The ${ENTRY_ID} variable is incorrectly substituted to an empty string.
Root Cause
In DefaultReplacementAppliedFunc (line ~199), when processing a braced variable with a default value, the function calls SubstituteWith() to process the remaining string:
if applied {
interpolatedNested, err := SubstituteWith(rest, mapping, pattern)
if err != nil {
return "", false, err
}
return value + interpolatedNested, true, nil
}The problem is that SubstituteWith() creates a new Config without the custom ReplacementFunc and WithoutLogging options. This causes the remaining variables to be processed with default behavior, which substitutes unset variables to empty strings.
Proposed Fix
Modify DefaultReplacementAppliedFunc to preserve the current config options when recursively processing the remaining string:
if applied {
// Preserve the current config options when processing the remaining string
options := []Option{
WithPattern(pattern),
}
if cfg.replacementFunc != nil {
options = append(options, WithReplacementFunction(cfg.replacementFunc))
}
if !cfg.logging {
options = append(options, WithoutLogging)
}
interpolatedNested, err := SubstituteWithOptions(rest, mapping, options...)
if err != nil {
return "", false, err
}
return value + interpolatedNested, true, nil
}Impact
This bug affects any application using compose-go/template with a custom ReplacementFunc, including:
- Telegraf (InfluxData's metrics collection agent)
- Any tool that wants to preserve unset variables for later substitution (e.g., two-stage variable expansion)
Workarounds
Until this is fixed, users can:
- Avoid mixing variables with and without defaults in the same string
- Ensure variables without defaults appear before variables with defaults
- Use
$VARsyntax instead of${VAR}for variables without defaults (uses different code path)
Additional Context
This issue was discovered while investigating why Telegraf's exec input plugin was not correctly handling environment variables. Users configure commands like:
[[inputs.exec]]
commands = ["sh -c '/script.py --file ${FILE:-/tmp/file.jpg} --id ${ENTRY_ID}'"]
environment = ["ENTRY_ID=12345"]The expected behavior is that ${ENTRY_ID} remains in the command string (for shell substitution at runtime), but it gets replaced with an empty string because it appears after ${FILE:-/tmp/file.jpg}.