Skip to content

feat(otlp): added tls server config option for http and grpc #2220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lucastemb wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat(otlp): added tls server config option for http and grpc #2220

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
93e1a74
feat(otlp): added tls server config option for http and grpc
lucastemb Jul 28, 2026
e4ef0f4
fix(otlp): added integration test and addressed codex comments
lucastemb Jul 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

88 changes: 48 additions & 40 deletions docs/agent-data-plane/configuration/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -430,46 +430,54 @@ ways that are not yet fully characterized.

The following settings are specific to ADP and have no equivalent in the core agent.

| Config Key | Description | Default |
| --------------------------------------------------------------- | ------------------------------------------ | -------------- |
| `aggregate_context_limit` | Max contexts per aggregation window | |
| `aggregate_flush_interval` | Aggregator flush period | |
| `aggregate_passthrough_idle_flush_timeout` | Passthrough buffer flush delay | |
| `aggregate_window_duration_seconds` | Aggregation window size | |
| `apm_config.obfuscation.sql.dbms` | SQL obfuscation DBMS dialect | |
| `apm_config.obfuscation.sql.dollar_quoted_func` | Preserve dollar-quoted SQL functions | |
| `apm_config.obfuscation.sql.keep_sql_alias` | Preserve SQL aliases in obfuscation | |
| `apm_config.obfuscation.sql.replace_digits` | Replace digits in SQL obfuscation | |
| `apm_config.obfuscation.sql.table_names` | Collect table names during obfuscation | |
| `counter_expiry_seconds` | Idle counter keep-alive duration | 300 |
| `data_plane.metrics.v3.series.enabled` | Enable ADP V3 series | false |
| `data_plane.otlp.receiver_grpc_endpoint_temporary` | ADP OTLP gRPC listen endpoint | localhost:6317 |
| `data_plane.otlp.receiver_http_endpoint_temporary` | ADP OTLP HTTP listen endpoint | localhost:6318 |
| `data_plane.serializer_zstd_compressor_level` | ADP zstd compression level | 3 |
| `data_plane.stop_timeout` | ADP graceful shutdown timeout (s) | derived |
| `dogstatsd_allow_context_heap_allocs` | Allow heap allocations for contexts | |
| `dogstatsd_autoscale_udp_listeners` | Bind multiple UDP sockets via SO_REUSEPORT | |
| `dogstatsd_buffer_count_max` | Max receive buffers | 256 |
| `dogstatsd_buffer_count` | Baseline receive buffers | 128 |
| `dogstatsd_cached_contexts_limit` | Max cached metric contexts | |
| `dogstatsd_cached_tagsets_limit` | Max cached tagsets | |
| `dogstatsd_mapper_string_interner_size` | Mapper string interner capacity | |
| `dogstatsd_minimum_sample_rate` | Floor for metric sample rates | |
| `dogstatsd_permissive_decoding` | Relaxes decoder strictness | true |
| `dogstatsd_string_interner_size_bytes` | Explicit byte budget for context interner | |
| `dogstatsd_tcp_port` | TCP listen port for DSD | |
| `flush_timeout_secs` | Encoder flush timeout (secs) | |
| `memory_limit` | Process memory limit | |
| `memory_slop_factor` | Memory accounting slop fraction | 0.25 |
| `otlp_allow_context_heap_allocs` | Allow heap allocations for OTLP contexts | |
| `otlp_cached_contexts_limit` | Max cached OTLP metric contexts | |
| `otlp_cached_tagsets_limit` | Max cached OTLP tagsets | |
| `otlp_config.receiver.protocols.http.transport` | OTLP HTTP receiver transport | |
| `otlp_config.traces.enable_otlp_compute_top_level_by_span_kind` | Enable OTLP top-level-by-span-kind | |
| `otlp_config.traces.ignore_missing_datadog_fields` | Ignore missing Datadog fields in OTLP | |
| `otlp_config.traces.string_interner_size` | OTLP trace string interner capacity | |
| `otlp_string_interner_size` | OTLP context interner capacity | |
| `serializer_max_metrics_per_payload` | Max metrics per payload | |
| Config Key | Description | Default |
| --------------------------------------------------------------- | --------------------------------------------------- | -------------- |
| `aggregate_context_limit` | Max contexts per aggregation window | |
| `aggregate_flush_interval` | Aggregator flush period | |
| `aggregate_passthrough_idle_flush_timeout` | Passthrough buffer flush delay | |
| `aggregate_window_duration_seconds` | Aggregation window size | |
| `apm_config.obfuscation.sql.dbms` | SQL obfuscation DBMS dialect | |
| `apm_config.obfuscation.sql.dollar_quoted_func` | Preserve dollar-quoted SQL functions | |
| `apm_config.obfuscation.sql.keep_sql_alias` | Preserve SQL aliases in obfuscation | |
| `apm_config.obfuscation.sql.replace_digits` | Replace digits in SQL obfuscation | |
| `apm_config.obfuscation.sql.table_names` | Collect table names during obfuscation | |
| `counter_expiry_seconds` | Idle counter keep-alive duration | 300 |
| `data_plane.metrics.v3.series.enabled` | Enable ADP V3 series | false |
| `data_plane.otlp.receiver_grpc_endpoint_temporary` | ADP OTLP gRPC listen endpoint | localhost:6317 |
| `data_plane.otlp.receiver_http_endpoint_temporary` | ADP OTLP HTTP listen endpoint | localhost:6318 |
| `data_plane.serializer_zstd_compressor_level` | ADP zstd compression level | 3 |
| `data_plane.stop_timeout` | ADP graceful shutdown timeout (s) | derived |
| `dogstatsd_allow_context_heap_allocs` | Allow heap allocations for contexts | |
| `dogstatsd_autoscale_udp_listeners` | Bind multiple UDP sockets via SO_REUSEPORT | |
| `dogstatsd_buffer_count_max` | Max receive buffers | 256 |
| `dogstatsd_buffer_count` | Baseline receive buffers | 128 |
| `dogstatsd_cached_contexts_limit` | Max cached metric contexts | |
| `dogstatsd_cached_tagsets_limit` | Max cached tagsets | |
| `dogstatsd_mapper_string_interner_size` | Mapper string interner capacity | |
| `dogstatsd_minimum_sample_rate` | Floor for metric sample rates | |
| `dogstatsd_permissive_decoding` | Relaxes decoder strictness | true |
| `dogstatsd_string_interner_size_bytes` | Explicit byte budget for context interner | |
| `dogstatsd_tcp_port` | TCP listen port for DSD | |
| `flush_timeout_secs` | Encoder flush timeout (secs) | |
| `memory_limit` | Process memory limit | |
| `memory_slop_factor` | Memory accounting slop fraction | 0.25 |
| `otlp_allow_context_heap_allocs` | Allow heap allocations for OTLP contexts | |
| `otlp_cached_contexts_limit` | Max cached OTLP metric contexts | |
| `otlp_cached_tagsets_limit` | Max cached OTLP tagsets | |
| `otlp_config.receiver.protocols.grpc.tls.ca_file` | OTLP gRPC TLS compatibility certificate authorities | |

@thieman thieman Jul 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the goal to fully implement the options from the OTEL side? Seems like we're missing quite a bit if so? https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/README.md

@lucastemb lucastemb Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe so. I scoped this PR to the options that are present here, but I am confused about what the authoritative and comprehensive list is.

@thieman thieman Jul 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My guess is that your link is outdated and that the .schema.yaml files strewn about are authoritative, but maybe we should check with someone on the OTEL side. Here's the schema for TLS: https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/config.schema.yaml

Is our actual goal to 100% support all OTEL config options?

| `otlp_config.receiver.protocols.grpc.tls.cert_file` | OTLP gRPC TLS server certificate chain | |
| `otlp_config.receiver.protocols.grpc.tls.client_ca_file` | OTLP gRPC mutual TLS client certificate authorities | |
| `otlp_config.receiver.protocols.grpc.tls.key_file` | OTLP gRPC TLS server private key | |
| `otlp_config.receiver.protocols.http.tls.ca_file` | OTLP HTTP TLS compatibility certificate authorities | |
| `otlp_config.receiver.protocols.http.tls.cert_file` | OTLP HTTP TLS server certificate chain | |
| `otlp_config.receiver.protocols.http.tls.client_ca_file` | OTLP HTTP mutual TLS client certificate authorities | |
| `otlp_config.receiver.protocols.http.tls.key_file` | OTLP HTTP TLS server private key | |
| `otlp_config.receiver.protocols.http.transport` | OTLP HTTP receiver transport | |
| `otlp_config.traces.enable_otlp_compute_top_level_by_span_kind` | Enable OTLP top-level-by-span-kind | |
| `otlp_config.traces.ignore_missing_datadog_fields` | Ignore missing Datadog fields in OTLP | |
| `otlp_config.traces.string_interner_size` | OTLP trace string interner capacity | |
| `otlp_string_interner_size` | OTLP context interner capacity | |
| `serializer_max_metrics_per_payload` | Max metrics per payload | |

### `data_plane.metrics.v3.series.enabled`

Expand Down
19 changes: 18 additions & 1 deletion lib/agent-data-plane-config-system/src/saluki_only.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,10 @@ use std::{num::NonZeroUsize, time::Duration};

use agent_data_plane_config::control::ListenAddress;
use agent_data_plane_config::defaults::{DEFAULT_STRING_INTERNER_SIZE_BYTES, MAX_STRING_INTERNER_SIZE_BYTES};
use agent_data_plane_config::domains::traces::{OttlErrorMode, OttlFilter, OttlTransform};
use agent_data_plane_config::domains::{
otlp::TlsConfig,
traces::{OttlErrorMode, OttlFilter, OttlTransform},
};
use agent_data_plane_config::SalukiConfiguration;
use bytesize::ByteSize;
use saluki_config::DurationString;
Expand Down Expand Up @@ -311,16 +314,28 @@ pub struct OtlpConfigReceiver {
#[derive(Clone, Debug, Default, Deserialize)]
#[serde(default)]
pub struct OtlpConfigReceiverProtocols {
/// OTLP gRPC receiver TLS knobs (`otlp_config.receiver.protocols.grpc.tls.*`).
pub grpc: OtlpConfigReceiverProtocolsGrpc,
/// OTLP HTTP receiver knobs (`otlp_config.receiver.protocols.http.*`).
pub http: OtlpConfigReceiverProtocolsHttp,
}

/// `otlp_config.receiver.protocols.grpc.*` values absent from the Datadog schema.
#[derive(Clone, Debug, Default, Deserialize)]
#[serde(default)]
pub struct OtlpConfigReceiverProtocolsGrpc {
/// OTLP gRPC receiver TLS configuration.
pub tls: TlsConfig,

@webern webern Aug 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that these are being added as saluki_only keys is suspicious. If these are not part of the Agent's configurable surface, i.e. if they are part of the OTEL collector but not configurable when running OTEL Ingest via the Agent, then they would be out of scope for the current project.

}

/// `otlp_config.receiver.protocols.http.*`.
#[derive(Clone, Debug, Default, Deserialize)]
#[serde(default)]
pub struct OtlpConfigReceiverProtocolsHttp {
/// OTLP HTTP receiver transport (`otlp_config.receiver.protocols.http.transport`).
pub transport: Option<String>,
/// OTLP HTTP receiver TLS configuration.
pub tls: TlsConfig,
}

fn deserialize_string_interner_size<'de, D>(deserializer: D) -> Result<NonZeroUsize, D::Error>
Expand Down Expand Up @@ -513,6 +528,8 @@ impl SalukiOnly {
if let Some(v) = self.otlp_config.receiver.protocols.http.transport.clone() {
otlp.receiver.http.transport = v;
}
otlp.receiver.grpc.tls = self.otlp_config.receiver.protocols.grpc.tls.clone();
otlp.receiver.http.tls = self.otlp_config.receiver.protocols.http.tls.clone();
otlp.traces.string_interner_size = self.otlp_config.traces.string_interner_size;
if let Some(v) = self.otlp_config.traces.enable_otlp_compute_top_level_by_span_kind {
otlp.traces.enable_compute_top_level_by_span_kind = v;
Expand Down
81 changes: 79 additions & 2 deletions lib/agent-data-plane-config/src/domains/otlp.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

use std::{num::NonZeroUsize, str::FromStr};

use serde::Serialize;
use serde::{Deserialize, Serialize};

use crate::defaults::DEFAULT_STRING_INTERNER_SIZE_BYTES;
use crate::Error;
Expand Down Expand Up @@ -81,6 +81,55 @@ impl FromStr for HistogramMode {
}
}

/// TLS settings for an inbound OTLP receiver.
#[derive(Clone, Debug, Default, Deserialize, PartialEq, Eq, Serialize)]
#[serde(default)]
pub struct TlsConfig {
Comment on lines +85 to +87

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Default omitted TLS fields during deserialization

When a user supplies the normal TLS configuration containing only cert_file and key_file, deserialization fails because every String in TlsConfig is required; the #[serde(default)] attributes on the surrounding protocol structs do not apply inside a present tls object. Add serde defaults to this struct or its fields so optional ca_file and client_ca_file values remain empty and basic server TLS can start.

Useful? React with 👍 / 👎.

@lucastemb lucastemb Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e4ef0f4

/// Path to PEM-encoded certificate authorities loaded for Collector configuration compatibility.
///
/// For an inbound receiver, this does not affect client authentication. Use [`Self::client_ca_file`] to configure
/// mutual TLS.
pub ca_file: String,

/// Path to the PEM-encoded certificate chain the receiver presents to clients.
pub cert_file: String,

/// Path to the PEM-encoded private key matching [`Self::cert_file`].
pub key_file: String,

/// Path to PEM-encoded certificate authorities trusted for client certificates.
///
/// When set, the receiver requires and verifies a client certificate, enabling mutual TLS.
pub client_ca_file: String,
}

impl TlsConfig {
/// Returns whether TLS is configured for the receiver.
///
/// TLS is disabled when no paths are configured. A configured server identity requires both `cert_file` and
/// `key_file`; configuring only one, or configuring either CA path without an identity, is an error.
pub fn is_configured(&self) -> Result<bool, Error> {
match (
self.cert_file.is_empty(),
self.key_file.is_empty(),
self.ca_file.is_empty(),
self.client_ca_file.is_empty(),
) {
(true, true, true, true) => Ok(false),
(false, false, _, _) => Ok(true),
(true, true, false, _) => Err(Error::new_without_source(
"`otlp_config.receiver.protocols.*.tls.ca_file` requires `cert_file` and `key_file`",
)),
(true, true, _, false) => Err(Error::new_without_source(
"`otlp_config.receiver.protocols.*.tls.client_ca_file` requires `cert_file` and `key_file`",
)),
_ => Err(Error::new_without_source(
"`otlp_config.receiver.protocols.*.tls.cert_file` and `key_file` must be configured together",
)),
}
}
}

/// How cumulative monotonic sums are reported.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize)]
pub enum CumulativeMonotonicMode {
Expand Down Expand Up @@ -220,6 +269,9 @@ pub struct GrpcReceiver {

/// Transport the gRPC receiver binds (for example, `tcp` or `unix`).
pub transport: String,

/// TLS settings for this inbound receiver.
pub tls: TlsConfig,
}

/// OTLP HTTP receiver.
Expand All @@ -231,6 +283,9 @@ pub struct HttpReceiver {
/// Transport the HTTP receiver binds (for example, `tcp` or `unix`). (not in Datadog Agent
/// config schema)
pub transport: String,

/// TLS settings for this inbound receiver.
pub tls: TlsConfig,
}

impl Default for HttpReceiver {
Expand All @@ -239,6 +294,7 @@ impl Default for HttpReceiver {
// Witnessed; overwritten during drive.
endpoint: String::new(),
transport: "tcp".to_string(),
tls: TlsConfig::default(),
}
}
}
Expand Down Expand Up @@ -331,7 +387,28 @@ impl Default for Contexts {

#[cfg(test)]
mod tests {
use super::{CumulativeMonotonicMode, InitialCumulativeMonotonicValue};
use super::{CumulativeMonotonicMode, InitialCumulativeMonotonicValue, TlsConfig};

#[test]
fn tls_config_requires_a_complete_server_identity() {
let mut config = TlsConfig::default();
assert!(!config.is_configured().expect("an empty TLS config should disable TLS"));

config.cert_file = "server.pem".to_string();
assert!(config.is_configured().is_err());

config.key_file = "server.key".to_string();
assert!(config.is_configured().expect("a certificate and key should enable TLS"));

config.cert_file.clear();
config.key_file.clear();
config.client_ca_file = "clients.pem".to_string();
assert!(config.is_configured().is_err());

config.client_ca_file.clear();
config.ca_file = "compatibility-ca.pem".to_string();
assert!(config.is_configured().is_err());
}

#[test]
fn cumulative_monotonic_mode_parses_known_values() {
Expand Down
Loading