Skip to content

Commit 1f25677

Browse files
committed
fix(config): coerce scalar config leaves the way the Agent does
The Agent never reads a setting as the type its YAML holds: GetBool, GetInt, GetFloat64, and GetString each cast the stored value through spf13/cast. Permissiveness is therefore a property of a leaf's declared type, not of the key, so `dogstatsd_port: "8125"` and a string-typed leaf written as a YAML boolean are configurations the Agent accepts. The generated Datadog source model took each leaf's JSON type literally, so those spellings failed deserialization. That aborts the strict startup gate, and at runtime it rejects the whole Agent snapshot, holding every other key at its last-known-good value. Port cast.To{Bool,Int64,Float64,String}E into cast_de and have codegen attach one by leaf type, keeping the schema's type as the field type. env_decode now shares those parsers, so one accept-set serves the file, the environment, and the Agent stream. A value the cast cannot convert stays a hard error rather than the Agent's silent zero value. Every generated field is classified, and an unrecognized type fails the build, so a schema change cannot quietly ship a leaf that rejects input the Agent accepts. This subsumes the overlay's `input_shape` metadata, whose one shape (a byte size written as an integer) is what a string leaf now accepts by type, so it and string_de are removed. Type-directed coercion cannot cover a setting whose values are a closed set rather than a type, so model `use_v3_api.series.enabled` as a `V3SeriesMode` enum. Its `FromStr` lists the spellings the Agent's evaluator interprets, the translator parses once instead of every consumer re-parsing a `String`, and an uninterpretable mode recovers to disabled — what the Agent routes on — while still recording a translation error for the strict gate.
1 parent ca31d5f commit 1f25677

18 files changed

Lines changed: 1184 additions & 402 deletions

File tree

.claude/skills/config-system/SKILL.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,19 @@ Reserve `#[serde(flatten)]` for a struct that genuinely groups several *top-leve
123123
example, the forwarder's `forwarder_*` retry settings). Name a Rust field after its canonical
124124
section rather than renaming it onto one.
125125

126+
## Scalar leaf coercion
127+
128+
`DatadogConfiguration` scalar leaves accept what the Agent's permissive casting accepts. Codegen
129+
attaches a `cast_de.rs` coercion per schema type. For example `dogstatsd_port: "8125.0"` is valid
130+
and coerced to the int `8125`. `1` or `"T"` is coerced to a `bool`.
131+
132+
## Documented enum settings
133+
134+
A `string` setting whose documentation names a closed set of values (`otlp_config.metrics.histograms.mode`)
135+
becomes an enum in `agent-data-plane-config` with `#[default]` on the schema default and a `FromStr`
136+
listing the accepted spellings. The translator parses it and records the error, recovering to the value
137+
the Agent uses.
138+
126139
## Saluki-only values
127140

128141
Values absent from the Datadog schema reach `SalukiConfiguration` through the `SalukiOnly` source

.vale/styles/config/vocabularies/technical/accept.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -151,7 +151,7 @@ configurability
151151
cooldown
152152
crypto
153153
deserializable
154-
deserializer
154+
deserializer(s?)
155155
downcasted
156156
upcasted
157157
env

lib/agent-data-plane-config-system/src/system.rs

Lines changed: 29 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -289,6 +289,7 @@ mod tests {
289289
use std::time::Duration;
290290

291291
use agent_data_plane_config::domains::dogstatsd::OriginTagCardinality;
292+
use agent_data_plane_config::shared::V3SeriesMode;
292293
use agent_data_plane_config::Provenance;
293294
use agent_data_plane_config::{Live, SalukiConfiguration};
294295
use datadog_agent_config::DatadogConfiguration;
@@ -362,8 +363,8 @@ mod tests {
362363
.await;
363364

364365
assert_eq!(
365-
system.config().shared.metrics_encoding.v3_series_mode.mode,
366-
"datadog_only"
366+
system.config().shared.metrics_encoding.v3_series_mode,
367+
V3SeriesMode::DatadogOnly
367368
);
368369

369370
agent_tx
@@ -406,14 +407,13 @@ mod tests {
406407
.unwrap();
407408

408409
await_config(&system, "the streamed metrics V3 routing configuration", |config| {
409-
config.shared.metrics_encoding.v3_series_mode.mode == "false"
410+
config.shared.metrics_encoding.v3_series_mode == V3SeriesMode::Disabled
410411
&& config
411412
.shared
412413
.metrics_encoding
413-
.v3_series_mode
414-
.endpoint_modes
414+
.v3_series_endpoint_modes
415415
.get("https://app.datadoghq.com")
416-
.is_some_and(|mode| mode == "true")
416+
== Some(&V3SeriesMode::Enabled)
417417
})
418418
.await;
419419

@@ -546,6 +546,29 @@ mod tests {
546546
assert_eq!(system.config().domains.dogstatsd.debug_log.log_file_max_size, 10485760);
547547
}
548548

549+
#[tokio::test]
550+
async fn standalone_loads_scalars_written_in_any_form_the_agent_casts() {
551+
// The Agent reads a setting by casting whatever its configuration holds to the accessor's
552+
// type, so a boolean written where the schema declares a string, or a quoted integer, is a
553+
// configuration it accepts. Each must reach the typed model instead of aborting the strict
554+
// startup gate.
555+
let system = standalone_system(
556+
Some(json!({
557+
"use_v3_api": { "series": { "enabled": true } },
558+
"dogstatsd_port": "8126",
559+
})),
560+
None,
561+
)
562+
.await
563+
.expect("scalars in Agent-castable forms boot");
564+
565+
assert_eq!(
566+
system.config().shared.metrics_encoding.v3_series_mode,
567+
V3SeriesMode::Enabled
568+
);
569+
assert_eq!(system.config().domains.dogstatsd.listeners.port, 8126);
570+
}
571+
549572
#[tokio::test]
550573
async fn translation_invalid_update_is_rejected_keeping_last_known_good() {
551574
let (system, agent_tx) =

lib/agent-data-plane-config-system/src/translators/datadog_translator.rs

Lines changed: 68 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ use agent_data_plane_config::domains::otlp::{
2727
CumulativeMonotonicMode, HistogramMode, InitialCumulativeMonotonicValue, SummaryMode,
2828
DEFAULT_GRPC_MAX_RECV_MSG_SIZE_MIB,
2929
};
30-
use agent_data_plane_config::shared::ForwarderHttpProtocol;
30+
use agent_data_plane_config::shared::{ForwarderHttpProtocol, V3SeriesMode};
3131
use agent_data_plane_config::{ConfigValue, SalukiConfiguration};
3232
use bytesize::ByteSize;
3333
use datadog_agent_config::{drive, DatadogConfigWitness, DatadogConfiguration, TranslateError, TranslateErrors};
@@ -73,6 +73,20 @@ impl<'a> DatadogTranslator<'a> {
7373
fn record_error(&mut self, error: TranslateError) {
7474
self.errors.push(error);
7575
}
76+
77+
/// Parses a V3 series mode, recovering to disabled rather than to the field default.
78+
///
79+
/// A strict startup rejects the recorded error, and any path that keeps going routes series to
80+
/// the older intake, which is what the Agent does with a mode it cannot interpret.
81+
fn parse_v3_series_mode(&mut self, key: &'static str, value: &str) -> V3SeriesMode {
82+
match value.parse() {
83+
Ok(mode) => mode,
84+
Err(error) => {
85+
self.record_error(TranslateError::new(key, error));
86+
V3SeriesMode::Disabled
87+
}
88+
}
89+
}
7690
}
7791

7892
/// Returns `None` for an empty `s`; otherwise returns `Some(s)`.
@@ -1114,18 +1128,22 @@ impl DatadogConfigWitness for DatadogTranslator<'_> {
11141128
}
11151129

11161130
fn consume_use_v3_api_series_enabled(&mut self, value: String) {
1117-
// TODO: consider modeling as an enum.
1118-
self.config.shared.metrics_encoding.v3_series_mode.mode = value;
1131+
let mode = self.parse_v3_series_mode("use_v3_api.series.enabled", &value);
1132+
self.config.shared.metrics_encoding.v3_series_mode = mode;
11191133
}
11201134

11211135
fn consume_use_v3_api_series_endpoints(&mut self, value: ::serde_json::Map<String, ::serde_json::Value>) {
1122-
self.config.shared.metrics_encoding.v3_series_mode.endpoint_modes = value
1136+
// The Agent accepts a per-endpoint mode written as any scalar, so a JSON boolean reaches the
1137+
// same parser as the string it renders to.
1138+
let modes: HashMap<String, V3SeriesMode> = value
11231139
.into_iter()
11241140
.map(|(endpoint, mode)| {
11251141
let mode = mode.as_str().map(str::to_string).unwrap_or_else(|| mode.to_string());
1142+
let mode = self.parse_v3_series_mode("use_v3_api.series.endpoints", &mode);
11261143
(endpoint, mode)
11271144
})
11281145
.collect();
1146+
self.config.shared.metrics_encoding.v3_series_endpoint_modes = modes;
11291147
}
11301148

11311149
fn consume_vector_metrics_enabled(&mut self, value: bool) {
@@ -1168,6 +1186,7 @@ mod tests {
11681186
CumulativeMonotonicMode, InitialCumulativeMonotonicValue, SummaryMode, DEFAULT_GRPC_MAX_RECV_MSG_SIZE_MIB,
11691187
},
11701188
};
1189+
use agent_data_plane_config::shared::V3SeriesMode;
11711190
use agent_data_plane_config::{ConfigValue, SalukiConfiguration};
11721191
use datadog_agent_config::{DatadogConfiguration, TranslateErrors};
11731192
use saluki_config::dynamic::{ConfigSetting, Provenance as StreamProvenance};
@@ -1314,6 +1333,51 @@ mod tests {
13141333
assert!(errors.to_string().contains("greater than or equal to 0"));
13151334
}
13161335

1336+
#[test]
1337+
fn v3_series_modes_translate_from_every_form_the_agent_accepts() {
1338+
let (config, errors) = translate_explicit(json!({
1339+
"use_v3_api": {
1340+
"series": {
1341+
"enabled": true,
1342+
"endpoints": { "https://app.datadoghq.com": "datadog_only", "https://opw.example.com": false },
1343+
}
1344+
}
1345+
}));
1346+
1347+
assert!(errors.is_none());
1348+
assert_eq!(config.shared.metrics_encoding.v3_series_mode, V3SeriesMode::Enabled);
1349+
assert_eq!(
1350+
config
1351+
.shared
1352+
.metrics_encoding
1353+
.v3_series_endpoint_modes
1354+
.get("https://app.datadoghq.com"),
1355+
Some(&V3SeriesMode::DatadogOnly)
1356+
);
1357+
assert_eq!(
1358+
config
1359+
.shared
1360+
.metrics_encoding
1361+
.v3_series_endpoint_modes
1362+
.get("https://opw.example.com"),
1363+
Some(&V3SeriesMode::Disabled)
1364+
);
1365+
}
1366+
1367+
#[test]
1368+
fn an_uninterpretable_v3_series_mode_disables_v3_and_records_a_translation_error() {
1369+
// The Agent routes to the older intake for a mode it cannot interpret, so the recovered value
1370+
// is disabled rather than the `datadog_only` field default.
1371+
let (config, errors) = translate_explicit(json!({
1372+
"use_v3_api": { "series": { "enabled": "sometimes" } }
1373+
}));
1374+
1375+
assert_eq!(config.shared.metrics_encoding.v3_series_mode, V3SeriesMode::Disabled);
1376+
let errors = errors.expect("an uninterpretable mode should record a translation error");
1377+
assert!(errors.to_string().contains("use_v3_api.series.enabled"));
1378+
assert!(errors.to_string().contains("unknown V3 series mode `sometimes`"));
1379+
}
1380+
13171381
// Issue #1965: the Core Agent streams `dd_url` at its schema default even when the operator
13181382
// configured only `site`. The translator used to compare the URL against that default and treat a
13191383
// match as unset, which also discarded an operator's deliberate choice of the default intake.

lib/agent-data-plane-config/src/shared.rs

Lines changed: 79 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,13 @@
22
33
use std::collections::HashMap;
44
use std::path::PathBuf;
5+
use std::str::FromStr;
56
use std::time::Duration;
67

78
use serde::Serialize;
89

910
use crate::defaults::DEFAULT_ENCODER_FLUSH_TIMEOUT;
10-
use crate::ConfigValue;
11+
use crate::{ConfigValue, Error};
1112

1213
/// Cross-cutting configuration shared across domains.
1314
#[derive(Clone, Debug, Default, PartialEq, Serialize)]
@@ -300,8 +301,12 @@ pub struct MetricsEncoding {
300301
/// V3 metrics-intake protocol settings (`serializer_experimental_use_v3_api.*`).
301302
pub v3_api: V3ApiEncoding,
302303

303-
/// Global and per-endpoint V3 series routing mode (`use_v3_api.series.*`).
304+
/// Global V3 series routing mode (`use_v3_api.series.enabled`).
304305
pub v3_series_mode: V3SeriesMode,
306+
307+
/// Per-endpoint V3 series routing overrides, keyed by endpoint URL
308+
/// (`use_v3_api.series.endpoints`).
309+
pub v3_series_endpoint_modes: HashMap<String, V3SeriesMode>,
305310
}
306311

307312
impl Default for MetricsEncoding {
@@ -321,6 +326,7 @@ impl Default for MetricsEncoding {
321326
histogram: HistogramEncoding::default(),
322327
v3_api: V3ApiEncoding::default(),
323328
v3_series_mode: V3SeriesMode::default(),
329+
v3_series_endpoint_modes: HashMap::new(),
324330
}
325331
}
326332
}
@@ -375,24 +381,33 @@ impl Default for V3ApiSettings {
375381
}
376382
}
377383

378-
/// Global and per-endpoint V3 series routing mode (`use_v3_api.series.*`).
379-
#[derive(Clone, Debug, PartialEq, Serialize)]
380-
pub struct V3SeriesMode {
381-
/// Global V3 series mode.
382-
///
383-
/// Defaults to `datadog_only`, which enables V3 only for configured Datadog intake URLs.
384-
/// TODO: consider modeling as an enum.
385-
pub mode: String,
384+
/// Whether series are routed to the V3 metrics intake (`use_v3_api.series.*`).
385+
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize)]
386+
pub enum V3SeriesMode {
387+
/// Route series to the V3 intake.
388+
Enabled,
389+
390+
/// Route series to the older intake.
391+
Disabled,
386392

387-
/// Per-endpoint V3 series mode overrides, keyed by endpoint URL.
388-
pub endpoint_modes: HashMap<String, String>,
393+
/// Route series to the V3 intake only for endpoints that are Datadog intake URLs.
394+
#[default]
395+
DatadogOnly,
389396
}
390397

391-
impl Default for V3SeriesMode {
392-
fn default() -> Self {
393-
Self {
394-
mode: "datadog_only".to_string(),
395-
endpoint_modes: HashMap::new(),
398+
impl FromStr for V3SeriesMode {
399+
type Err = Error;
400+
401+
// The Agent reads this setting as a string and then interprets it, accepting more spellings than
402+
// `strconv.ParseBool` does, so the accepted set is wider than that of a `boolean` leaf.
403+
fn from_str(value: &str) -> Result<Self, Self::Err> {
404+
match value.trim().to_ascii_lowercase().as_str() {
405+
"true" | "1" | "t" | "yes" | "on" => Ok(Self::Enabled),
406+
"false" | "0" | "f" | "no" | "off" | "" => Ok(Self::Disabled),
407+
"datadog_only" => Ok(Self::DatadogOnly),
408+
other => Err(Error::new_without_source(format!(
409+
"unknown V3 series mode `{other}`; expected a boolean or `datadog_only`"
410+
))),
396411
}
397412
}
398413
}
@@ -438,3 +453,50 @@ pub struct AutoscalingFailover {
438453
/// Metrics designated for failover.
439454
pub metrics: Vec<String>,
440455
}
456+
457+
#[cfg(test)]
458+
mod tests {
459+
use super::V3SeriesMode;
460+
461+
#[test]
462+
fn v3_series_mode_parses_every_form_the_agent_interprets() {
463+
for (value, expected) in [
464+
("true", V3SeriesMode::Enabled),
465+
("TRUE", V3SeriesMode::Enabled),
466+
("1", V3SeriesMode::Enabled),
467+
("t", V3SeriesMode::Enabled),
468+
("yes", V3SeriesMode::Enabled),
469+
("on", V3SeriesMode::Enabled),
470+
("false", V3SeriesMode::Disabled),
471+
("0", V3SeriesMode::Disabled),
472+
("f", V3SeriesMode::Disabled),
473+
("no", V3SeriesMode::Disabled),
474+
("off", V3SeriesMode::Disabled),
475+
("", V3SeriesMode::Disabled),
476+
(" datadog_only ", V3SeriesMode::DatadogOnly),
477+
] {
478+
assert_eq!(
479+
value.parse::<V3SeriesMode>().expect("mode should parse"),
480+
expected,
481+
"{value}"
482+
);
483+
}
484+
}
485+
486+
#[test]
487+
fn v3_series_mode_rejects_an_uninterpretable_value() {
488+
let error = "sometimes"
489+
.parse::<V3SeriesMode>()
490+
.expect_err("an uninterpretable mode should be rejected");
491+
492+
assert_eq!(
493+
error.to_string(),
494+
"unknown V3 series mode `sometimes`; expected a boolean or `datadog_only`"
495+
);
496+
}
497+
498+
#[test]
499+
fn v3_series_mode_defaults_to_datadog_only() {
500+
assert_eq!(V3SeriesMode::default(), V3SeriesMode::DatadogOnly);
501+
}
502+
}

lib/datadog-agent/config-overlay-model/src/lib.rs

Lines changed: 0 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -59,9 +59,6 @@ pub struct FullSupport {
5959
/// GitHub issue tracking number.
6060
#[serde(default)]
6161
pub issue: Option<String>,
62-
/// Accepted input shape when it is wider than the schema's declared type (see [`InputShape`]).
63-
#[serde(default)]
64-
pub input_shape: Option<InputShape>,
6562
/// Fields to support the `config_registry` and configuration smoke tests.
6663
pub test_support: TestSupport,
6764
}
@@ -82,9 +79,6 @@ pub struct PartialSupport {
8279
/// GitHub issue tracking number.
8380
#[serde(default)]
8481
pub issue: Option<String>,
85-
/// Accepted input shape when it is wider than the schema's declared type (see [`InputShape`]).
86-
#[serde(default)]
87-
pub input_shape: Option<InputShape>,
8882
/// Fields to support the `config_registry` and configuration smoke tests.
8983
pub test_support: TestSupport,
9084
}
@@ -241,19 +235,6 @@ pub enum ValueType {
241235
StringList,
242236
}
243237

244-
/// A widened input shape a schema `string` leaf accepts beyond a bare string.
245-
///
246-
/// The vendored schema types some settings as `string` but documents an equivalent numeric form
247-
/// (for example a byte size given as `10485760` instead of `"10MB"`). The schema cannot express that
248-
/// union, so the overlay names it and codegen attaches a tolerant deserializer to the generated
249-
/// field.
250-
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize)]
251-
#[serde(rename_all = "snake_case")]
252-
pub enum InputShape {
253-
/// Accept a string unchanged, or a non-negative integer normalized to its decimal string.
254-
StringOrInteger,
255-
}
256-
257238
/// File paths to the two YAML files required as input by this library.
258239
///
259240
/// Defaults to the canonical location of the required schema files in this library.

0 commit comments

Comments
 (0)