Fix ExpandedValue sanitization on struct collection #14413

theomagellan · 2026-01-12T16:00:14Z

Description

useExpandedValue would resolve ExpandedValues after checking if a collection's type was "stringy." For collections of structs, isStringy always returns false, causing all ExpandedValues to be sanitized to their parsed values and breaking decoding of string fields if the parsed value was not string.

Now, useExpandedValue checks if it's dealing with a collection of structs; in which case, skip sanitization and rely on mapstructure's per-field decoding.

Testing

Added tests relying on configopaque.MapList which is an alias on the struct collection []Pair

… unmarshalling a confmap `useExpandedValue` would resolve ExpandedValues after checking if a collection's type was "stringy." For collections of structs, `isStringy` always returns false, causing all ExpandedValues to be sanitized to their parsed values and breaking decoding of string fields if the parsed value was not `string`. Now, `useExpandedValue` checks if it's dealing with a collection of structs; in which case, skip sanitization and rely on mapstructure's per-field decoding.

…n-on-struct-collection

theomagellan · 2026-01-12T16:27:11Z

As discussed with @mx-psi, I'm kindly pinging @jade-guiton-dd to this PR!

codecov · 2026-01-13T09:01:43Z

Codecov Report

❌ Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.79%. Comparing base (97f66a9) to head (80a019a).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
confmap/internal/decoder.go	0.00%	5 Missing and 1 partial ⚠️

❌ Your patch status has failed because the patch coverage (0.00%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #14413      +/-   ##
==========================================
- Coverage   91.83%   91.79%   -0.04%     
==========================================
  Files         677      677              
  Lines       42679    42680       +1     
==========================================
- Hits        39195    39180      -15     
- Misses       2427     2439      +12     
- Partials     1057     1061       +4

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

jade-guiton-dd · 2026-01-14T16:59:05Z

Hello 👋 Sorry, even after reading the existing code and your PR, I still don't really understand what the problem was. Could you expand on:

causing all ExpandedValues to be sanitized to their parsed values and breaking decoding of string fields if the parsed value was not string.

Reading the code, it seems to me that an array of structs should already be kept as-is by the sanitization operation. Did I miss something?

theomagellan · 2026-01-14T17:23:11Z

Reading the code, it seems to me that an array of structs should already be kept as-is by the sanitization operation. Did I miss something?

The issue used to happen in cases like what this test does.

When decoding an ExpandedValue inside a struct field in a collection, the code would decode the ExpandedValue as its Value field (of any type) because we were deeming that a struct type was not stringy.
This would break cases like mentioned above where we would, later on in the unmarshalling process, try to set the result of our decoded ExpandedValue inside a struct field that would have been decoded as stringy if the decoding operation would've been delayed.

Commenting the fix and running the added tests fails with the error:

FAIL: TestMapListWithExpandedValueIntValue (0.00s)
[...]
'headers[0].value' expected type 'configopaque.String', got unconvertible type 'int'

Since the ExpandedValue was decoded as its Value because useExpandedValue called isStringyStructure on the type of the collection (collection is []Pair, so the type is Pair) which is not a stringy type.

Sorry if I wasn't clear enough before, I hope my explanations help!

jade-guiton-dd · 2026-01-14T17:38:03Z

Ah, it seems there was a misunderstanding on my part (isStringyStructure checks the type of the destination, whereas sanitizeExpanded switches over the type of the source). The PR makes sense to me now.

One thing I thought about is that, in the same way your PR delegates to mapstructure the task of inspecting the struct so we can more accurately tell when we're assigning ExpandedValues to strings, have you tried completely deleting the switch to.Kind() { blocks, so that arrays/slices/maps all get mapped by mapstructure instead of the sanitize functions, and we just rely on the initial test to identify values to keep as strings? It's possible there's an edge case I'm not thinking of where that wouldn't work, but I feel like that would be worth a try to simplify the logic instead of making it more complex.

theomagellan · 2026-01-14T17:54:24Z

I tested it and it also solves the issue and doesn't break any other tests.
My understanding of the code was that these checks were here for performance improvements, as for cases like []string we can avoid subsequent hook calls since we already know that the collection is only composed of stringy elements.

I do agree that removing the switch altogether would make the code simpler and would also resolve any edge cases like the one I brought up. What do you think?

jade-guiton-dd · 2026-01-15T13:47:15Z

Regardless of Pablo's intent 2 years ago, I suspect it's not actually an optimization: I think mapstructure will walk the sanitized slice/map returned by the hook and call it again on each contained items regardless. So I don't think removing the switch should cause a performance regression; I'd say it's an unconditional improvement.

However, I'm not 100% confident that this won't break someone somewhere (mapstructure's behavior and the interactions between our hooks can be a bit arcane in my experience), so I think it may be prudent to introduce a beta feature gate and run the latter part of the function if the gate has been disabled. That way we have a quick solution if someone encounters breakage; and if we don't receive any complaints, we can stabilize the gate and fully remove the code in a few releases.

theomagellan · 2026-01-15T14:34:29Z

confmap/internal/featuregates.go

+	featuregate.StageBeta,
+	featuregate.WithRegisterFromVersion("v0.144.0"),
+	featuregate.WithRegisterDescription("Disables early sanitization of ExpandedValue during config unmarshalling, allowing mapstructure to handle type conversion at the field level. Fixes decoding errors when environment variable values are parsed as non-string types (e.g., numbers, booleans) but need to be assigned to string fields."),
+	featuregate.WithRegisterReferenceURL("https://github.com/open-telemetry/opentelemetry-collector/pull/14413#issuecomment-3754949484"),


All other featuregates reference issues directly instead of PRs.
Should I create an issue or is this good enough? Sorry.

I think referencing a PR should be fine.

theomagellan · 2026-01-15T14:35:41Z

confmap/internal/decoder.go

+				// converted based on its target field type.
+				if elemType.Kind() == reflect.Struct {
+					return data, nil
+				}


I still kept the original fix inside the old behavior since it's still required to fix the problem I encountered.
I'm happy to remove it if you think it's unnecessary.

Hm, I doubt the original fix would cause issues, but out of caution, I'd rather have disabling the feature gate return to the previous behavior, bug included. I figure that if someone's setup is broken by the new fix, I think there's a reasonable chance they will be broken by the original fix as well.

mx-psi · 2026-01-15T16:35:53Z

Regardless of Pablo's intent 2 years ago, I suspect it's not actually an optimization: I think mapstructure will walk the sanitized slice/map returned by the hook and call it again on each contained items regardless. So I don't think removing the switch should cause a performance regression; I'd say it's an unconditional improvement.

To be clear although I don't remember exactly what I was thinking at that point I don't think my intent here was performance-related, it was probably more just an oversight/something behavior-related.

However, I'm not 100% confident that this won't break someone somewhere (mapstructure's behavior and the interactions between our hooks can be a bit arcane in my experience), so I think it may be prudent to introduce a beta feature gate and run the latter part of the function if the gate has been disabled. That way we have a quick solution if someone encounters breakage; and if we don't receive any complaints, we can stabilize the gate and fully remove the code in a few releases.

I agree with this, mapstructure is always arcane!

- revert to old behavior when feature gate disabled, bug included - test unmarshalling with both behaviors and expect error when feature gate disabled

jade-guiton-dd

Looks good, just need to run make gotidy and add a change log entry (make chlog-new).

Codecov is claiming the old code in the if is completely untested, so I would also recommend confirming with a debugger whether the old code is being reached in the latter half of TestMapListWithExpandedValueIntValue. It could just be an issue with Codecov however.

confmap/internal/featuregates.go

theomagellan · 2026-01-16T12:30:45Z

confmap/internal/e2e/maplist_expanded_test.go

+}
+
+// TestStringyStructureWithExpandedValue tests the isStringyStructure path in useExpandValue
+func TestStringyStructureWithExpandedValue(t *testing.T) {


With this new test, we should now cover all cases in both the new and old behavior.

- renamed featuregate - added test with a stringy collection - added changelog entry Co-authored-by: Jade Guiton <jade.guiton@datadoghq.com>

The e2e test module was not recording coverage for the parent `confmap/internal` package that it tests. This caused test coverage to appear as 0% even though the tests exercise the code. This adds the parent package to COVER_PKGS so coverage of `confmap/internal` is properly tracked for e2e tests.

theomagellan · 2026-01-16T16:37:07Z

confmap/internal/e2e/Makefile

 include ../../../Makefile.Common
+
+# Override COVER_PKGS to include the parent package that this e2e module tests
+COVER_PKGS := go.opentelemetry.io/collector/confmap/internal,$(COVER_PKGS)


This seems to be needed since I am covering code in confmap/internal from the e2e module.
This feels out of scope of the PR to me and I would appreciate any suggestions on how we could handle this coverage issue!

I think we can do this in a separate PR if you are up to contributing it. In any case it's not a requirement for merging this PR (codecov is not a required check)

Thank you. I will revert this commit and open a new PR.
In the meantime, do you have any ideas why the changelog validate is unhappy? I don't really understand the issue here.

I think that's an issue from other changelogs on main, so no worries.

This reverts commit 7eef7e1.

…lection

.chloggen/fix-expandedvalue-sanitization-on-struct-collection.yaml

mx-psi

Almost there!

.chloggen/fix-expandedvalue-sanitization-on-struct-collection.yaml

Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>

mx-psi · 2026-01-20T12:50:17Z

Can you solve the merge conflict? Once you have dealt with that I can merge it

…lection

otelbot · 2026-01-20T15:49:52Z

Thank you for your contribution @theomagellan! 🎉 We would like to hear from you about your experience contributing to OpenTelemetry by taking a few minutes to fill out this survey.

#### Description The e2e test module was not recording coverage for the parent `confmap/internal` package that it tests. This caused test coverage to appear as 0% even though the tests exercise the code. This adds the parent package to `COVER_PKGS` so coverage of `confmap/internal` is properly tracked for e2e tests. When looking at overall coverage of `confmap/internal/decoder.go`, it goes from 91.4% to 96.3% of statements. #### Link to tracking issue Issue was found in PR #14413 (review).  #### Testing Running `make gotest-with-cover` without the fix, then looking at coverage in percentage with ```bash go tool covdata percent -i=./coverage/unit -pkg=go.opentelemetry.io/collector/confmap/internal ``` Running these commands again with the fix added shows the improvement. The e2e tests can also be run individually to verify they're tracking parent package coverage: ```bash make -C confmap/internal/e2e test-with-cover # [...] coverage: 45.3% of statements in go.opentelemetry.io/collector/confmap/internal, go.opentelemetry.io/collector/confmap/internal/e2e, ) ```

theomagellan and others added 2 commits January 8, 2026 18:07

Merge branch 'open-telemetry:main' into fix-expandedvalue-sanitizatio…

64dd47b

…n-on-struct-collection

theomagellan marked this pull request as ready for review January 12, 2026 16:21

theomagellan requested review from a team, evan-bradley and mx-psi as code owners January 12, 2026 16:21

lint

959d5a9

added featuregate

143c270

theomagellan commented Jan 15, 2026

View reviewed changes

review suggestions:

a74434c

- revert to old behavior when feature gate disabled, bug included - test unmarshalling with both behaviors and expect error when feature gate disabled

jade-guiton-dd reviewed Jan 16, 2026

View reviewed changes

confmap/internal/featuregates.go Outdated Show resolved Hide resolved

theomagellan force-pushed the fix-expandedvalue-sanitization-on-struct-collection branch from 0e6ef84 to 3f49486 Compare January 16, 2026 12:28

theomagellan commented Jan 16, 2026

View reviewed changes

theomagellan force-pushed the fix-expandedvalue-sanitization-on-struct-collection branch from 3f49486 to 68015e6 Compare January 16, 2026 12:50

suggestions:

97efc61

- renamed featuregate - added test with a stringy collection - added changelog entry Co-authored-by: Jade Guiton <jade.guiton@datadoghq.com>

theomagellan force-pushed the fix-expandedvalue-sanitization-on-struct-collection branch from 68015e6 to 97efc61 Compare January 16, 2026 13:09

theomagellan commented Jan 16, 2026

View reviewed changes

theomagellan and others added 2 commits January 16, 2026 16:49

Revert "fix(confmap): Include parent package in e2e test coverage"

8597240

This reverts commit 7eef7e1.

Merge branch 'main' into fix-expandedvalue-sanitization-on-struct-col…

d77eb64

…lection

mx-psi reviewed Jan 19, 2026

View reviewed changes

.chloggen/fix-expandedvalue-sanitization-on-struct-collection.yaml Outdated Show resolved Hide resolved

mx-psi reviewed Jan 19, 2026

View reviewed changes

.chloggen/fix-expandedvalue-sanitization-on-struct-collection.yaml Outdated Show resolved Hide resolved

.chloggen/fix-expandedvalue-sanitization-on-struct-collection.yaml Outdated Show resolved Hide resolved

theomagellan and others added 2 commits January 19, 2026 14:11

enhanced change log entry

4ce2ced

Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>

changed issue description in changelog

b28a536

jade-guiton-dd approved these changes Jan 19, 2026

View reviewed changes

Merge branch 'main' into fix-expandedvalue-sanitization-on-struct-col…

80a019a

…lection

mx-psi enabled auto-merge January 20, 2026 15:09

mx-psi added this pull request to the merge queue Jan 20, 2026

Merged via the queue into open-telemetry:main with commit 246e428 Jan 20, 2026
60 of 61 checks passed

theomagellan mentioned this pull request Jan 21, 2026

fix(confmap): Include parent package in e2e test coverage #14439

Merged

Fix ExpandedValue sanitization on struct collection #14413

Fix ExpandedValue sanitization on struct collection #14413

Uh oh!

Conversation

theomagellan commented Jan 12, 2026

Description

Testing

Uh oh!

theomagellan commented Jan 12, 2026

Uh oh!

codecov bot commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jade-guiton-dd commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theomagellan commented Jan 14, 2026

Uh oh!

jade-guiton-dd commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theomagellan commented Jan 14, 2026

Uh oh!

jade-guiton-dd commented Jan 15, 2026

Uh oh!

theomagellan Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

jade-guiton-dd Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

theomagellan Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jade-guiton-dd Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

mx-psi commented Jan 15, 2026

Uh oh!

jade-guiton-dd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

theomagellan Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

theomagellan Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

mx-psi Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

theomagellan Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

jade-guiton-dd Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mx-psi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mx-psi commented Jan 20, 2026

Uh oh!

Uh oh!

otelbot bot commented Jan 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Jan 13, 2026 •

edited

Loading

jade-guiton-dd commented Jan 14, 2026 •

edited

Loading

jade-guiton-dd commented Jan 14, 2026 •

edited

Loading

theomagellan Jan 15, 2026 •

edited

Loading

jade-guiton-dd Jan 16, 2026 •

edited

Loading