Commit 45626b1
committed
fix(config): permissively coerce scalar config leaves (#2325)
## Human Summary
In #2317 @aqian01 detected a flaw in the typed config deserialization mechanism. The ported, permissive, Viper type coercions that the Agent uses were only being applied to environment variables. This means that when we parsed a config file ourselves, we would not have applied those permissive coercions. This is now fixed so that we more completely mirror Agent config parsing behavior when migrating to typed config.
## AI Summary
The Datadog Agent accepts configuration values through `cast`, so a leaf's declared type determines which alternate spellings it accepts. `dogstatsd_port: "8125"` and a schema-`string` leaf written as a YAML boolean (`use_v3_api.series.enabled: true`, read with `GetString`) are both valid Agent configuration. ADP's generated model instead deserialized each leaf strictly according to its schema type, causing startup translation to fail or rejecting the entire runtime configuration snapshot.
This change:
- Adds serde deserializers for boolean, integer, float, and string schema scalar types.
- Adds a build-time `permissivize` pass that applies the appropriate deserializer to every generated scalar leaf based on its generated Rust type.
- Makes `env_decode` use the same parsers, aligning file, environment, and configuration-stream inputs.
- Fails code generation for unrecognized leaf types, so new schema types require an explicit coercion decision.
- Retains hard errors for values that cannot be converted, rather than accepting `cast`'s zero values.
- Removes the now-unnecessary `input_shape: string_or_integer` metadata and related types.
Field types remain unchanged; only deserialization behavior is permissive. Numeric strings are accepted in decimal form only. The generated model diff adds deserializer attributes, and the change covers the full generated scalar model rather than individual keys.
## Change Type
- [x] Bug fix
## How did you test this PR?
- Unit tests cover accepted and rejected spellings for each scalar type.
- A schema-driven test exercises every scalar leaf with a castable value and verifies the coerced value through a serialization round-trip.
- Translation-gate tests cover a boolean on the V3 series mode string leaf and a quoted `dogstatsd_port`.
- `make build-schema-overlay`, `cargo check --workspace --tests`, and `cargo clippy --workspace --tests` pass; config crate tests pass.
## References
- Related: #2317
---
## Note: #2317 merged into the wrong base
I merged #2317 into `m/confra-cast` (this PR) instead of `main`. This PR now carries both changes. The #2317 description follows verbatim.
---
# Merged from #2317: fix(config): migrate endpoints to typed config for provenance awareness
## Human Summary
This migrates all components affected by #1965 (we were unable to detect explicitly set to default vs default config values) to typed config to pick up the fix provided in #2279.
After implementation I ran a clean-room Opus audit with this prompt:
> please do a clean-room audit of this commit `f4ae503e807b398ca55ee3f6275f554889cff645`
>
> Our goal is to correct a previous incorrect behavior in which we were not sensitive to whether a configuration value was set by the Agent as a default or whether the customer explicitly set the value (and happened to set it to the default value).
>
> Other than that specific intentional bug fix, no other behavioral changes are intended.
>
> Validate that only the desired behavioral change has occurred and that configuration defaults are in-tact.
>
> Read AGENTS.md and /config-system
It only found two actionable items, which I fixed. One was that we were previously rejecting empty API keys, so I retained that behavior. The other was that an MRF endpoint consisting only of whitespace could be treated as a valid setting. Also fixed. All defaults stood up to scrutiny.
Edit: @aqian01 found a flaw that went undetected which became #2325.
### Review Guide
I tried breaking this up, which is why there are a bunch of subissues being closed all at once, but it did not divide very well. As such, the diff a bit large:
Focus on these files:
- `lib/saluki-components/*`
- `bin/agent-data-plane/src/cli/run.rs`
- `lib/agent-data-plane-config*`
Deprioritize generated code and inventory churn in:
- `lib/datadog-agent/*`
## AI Summary
Migrate Datadog endpoint, retry, proxy, forwarder, metrics encoder, MRF, and Cluster Agent configuration consumers from raw `GenericConfiguration` deserialization to resolved typed configuration.
Primary endpoint resolution now happens once in the typed configuration layer:
- A default-sourced `dd_url` no longer shadows `site`.
- An explicitly configured `dd_url`, including the schema-default URL, remains an override.
- MRF and Cluster Agent destination overrides cannot be overwritten by global endpoint settings.
- Retry queue size precedence uses configuration provenance, so explicit zero values are preserved.
- Raw configuration access remains only for live API-key refresh, secrets retry behavior, and the `run_path` compatibility fallback whose schema default is still an unresolved placeholder.
The schema inventories, generated registries, smoke-test metadata, and component tests are updated for the typed consumers.
## Change Type
- [x] Non-functional (chore, refactoring, docs)
## How did you test this PR?
- unit tests updated and created throughout
- integration test modified to test this behavior. this was proven to have caught the original bug (`red -> green`)
## References
- Closes: #1965
- Closes: #2310
- Closes: #2312
- Closes: #2313
- Closes: #2314
- Closes: #2315
- Closes: #2316
- Related to: #2279
Co-authored-by: matt.briggs <matt.briggs@datadoghq.com> 79f5b0c1 parent b9138cc commit 45626b1
1,774 files changed
Lines changed: 20645 additions & 20486 deletions
File tree
- agent-data-plane
- configuration
- api-docs
- agent_data_plane_config_system
- agent_data_plane_config
- defaults
- domains/multi_region_failover
- datadog_agent_config_overlay_model
- saluki_keys
- smoke_test_support
- datadog_agent_config_testing
- config_registry/structs
- datadog_agent_config
- cast_de
- classifier
- structs
- env_decode
- saluki_components
- config/mrf
- encoders
- forwarders
- saluki_core/data_model/event/metric
- search.index
- alias
- desc
- entry
- function
- generic_inverted_index
- name
- normalizedName
- path
- type
- src
- agent_data_plane_config_system
- translators
- agent_data_plane_config
- domains
- datadog_agent_config_overlay_model
- datadog_agent_config_testing
- config_registry
- datadog_agent_config
- classifier
- generated
- saluki_components
- common/datadog
- config
- encoders/datadog/metrics
- forwarders
- cluster_agent
- datadog
- transforms/mrf_gateway
- trait.impl
- core
- clone
- cmp
- convert
- default
- fmt
- iter/traits/collect
- marker
- panic/unwind_safe
- str/traits
- serde_core
- de
- ser
- assets
- chunks
- development
- reference
- adrs
- records
- architecture
- proposals
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
| 12 | + | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
| 19 | + | |
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
| 12 | + | |
13 | 13 | | |
14 | | - | |
| 14 | + | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
27 | | - | |
| 26 | + | |
| 27 | + | |
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
0 commit comments