Configuration errors are useless when flattened fields are involved. #568
Description
Context
Our error messages from saluki-config/figment are generally pretty good, and can pinpoint which field had an error, and why (expected type A but got type B, etc)... but this falls down when flattened fields are involved. Flattened fields are a serde idiom where a struct field can be a complex type on the Rust side, but is treated as if the struct's fields were all present at the same level of the struct field.
Due to how flattening is involved, all error information is erased, which means serde doesn't report it and saluki-config/figment can't do anything to get it. In turn, you get bad error messages like this:
2025-03-24 13:56:34 UTC | DATAPLANE | ERROR | (bin/agent-data-plane/src/main.rs:79) | Failed to configure Datadog Metrics destination.
Caused by:
Expected value for field '' to be 'u64', got 'float `15728640`' instead.
We should likely just flatten these structs ourselves. It'll make the code uglier, which is unfortunate... but better to have sane error messages. In the future, work like serde-rs/serde#2912 might enable serde implementations to properly capture path data when flattening is used, giving us a way to go back to a clean and flattened struct with proper error messages.