Skip to content

Latest commit

 

History

History
799 lines (646 loc) · 73 KB

File metadata and controls

799 lines (646 loc) · 73 KB

Agent Data Plane configuration

Agent Data Plane (ADP) implements data processing pipelines in Rust for better resource guarantees and efficiency. Because its architecture differs from the core Agent, certain configuration values may behave differently, be planned but not yet implemented, or not apply at all. This page documents those nuances.

ADP is designed to be transparent: customers configure data processing pipelines the same way they configure the core Agent. The sections below call out the cases where that is either not yet true, or not quite possible.

If you find an error on this page, please open an issue.

Unsupported Settings

Being Worked On

The following settings are not yet supported in ADP but are planned with GitHub issue links for tracking.

Config Key Description Issue
dogstatsd_experimental_http.enabled Enable experimental HTTP/H2C DSD listener #1682
dogstatsd_experimental_http.listen_address Bind address for experimental HTTP DSD listener #1682
tls_handshake_timeout HTTP TLS handshake timeout #178

Not Planned

The following settings exist in the core agent but are not planned for ADP, typically because ADP's architecture is fundamentally different or the feature is platform-specific.

Config Key Description Reason
aggregator_buffer_size Channel buffer depth for aggregator queues See below
aggregator_flush_metrics_and_serialize_in_parallel_buffer_size Parallel flush: series/sketch buffer size See below
aggregator_flush_metrics_and_serialize_in_parallel_chan_size Parallel flush: channel size See below
aggregator_use_tags_store Enable shared tag deduplication store See below
config_id Fleet Automation config ID tag Core Agent uses this only on Agent HA telemetry metrics.
data_plane.telemetry_enabled ADP telemetry toggle See below
data_plane.telemetry_listen_addr ADP telemetry listen address See below
dogstatsd_host_socket_path Host UDS socket dir for DSD Not read by DSD server; admission controller only.
dogstatsd_mem_based_rate_limiter.enabled Memory-based rate limiter toggle See below
dogstatsd_mem_based_rate_limiter.go_gc Memory rate limiter GC percent Go GC-specific; ADP uses memory_limit instead.
dogstatsd_mem_based_rate_limiter.high_soft_limit Memory rate limiter high soft limit Go GC-specific; ADP uses memory_limit instead.
dogstatsd_mem_based_rate_limiter.low_soft_limit Memory rate limiter low soft limit Go GC-specific; ADP uses memory_limit instead.
dogstatsd_mem_based_rate_limiter.memory_ballast Memory rate limiter heap ballast size Go GC-specific; ADP uses memory_limit instead.
dogstatsd_mem_based_rate_limiter.rate_check.factor Memory rate limiter check factor Go GC-specific; ADP uses memory_limit instead.
dogstatsd_mem_based_rate_limiter.rate_check.max Memory rate limiter check max interval Go GC-specific; ADP uses memory_limit instead.
dogstatsd_mem_based_rate_limiter.rate_check.min Memory rate limiter check min interval Go GC-specific; ADP uses memory_limit instead.
dogstatsd_mem_based_rate_limiter.soft_limit_freeos_check.factor Memory rate limiter FreeOS check factor Go GC-specific; ADP uses memory_limit instead.
dogstatsd_mem_based_rate_limiter.soft_limit_freeos_check.max Memory rate limiter FreeOS check max Go GC-specific; ADP uses memory_limit instead.
dogstatsd_mem_based_rate_limiter.soft_limit_freeos_check.min Memory rate limiter FreeOS check min Go GC-specific; ADP uses memory_limit instead.
dogstatsd_no_aggregation_pipeline_batch_size No-aggregation pipeline batch size Fixed in ADP topology.
dogstatsd_packet_buffer_flush_timeout Packet buffer flush timeout ADP decodes inline.
dogstatsd_packet_buffer_size Datagrams per packet buffer ADP decodes inline.
dogstatsd_pipeline_autoadjust Auto-adjust pipeline workers ADP uses async tasks.
dogstatsd_pipeline_count Parallel processing pipelines ADP uses async tasks.
dogstatsd_queue_size Packet channel buffer size ADP uses async tasks.
dogstatsd_stats_buffer Internal stats buffer size ADP does not expose the core agent's packet-per-second expvar endpoint, so there is no persistent stats endpoint buffer to configure.
dogstatsd_stats_enable Enable internal stats endpoint See below
dogstatsd_stats_port Internal stats endpoint port ADP does not expose the core agent's packet-per-second expvar endpoint, so dogstatsd_stats_port has no effect.
dogstatsd_telemetry_enabled_listener_id Per-listener telemetry tagging Not feasible to thread listener identity through ADP's async decode pipeline.
dogstatsd_workers_count Number of DSD processing workers ADP uses async tasks.
enable_json_stream_shared_compressor_buffers Pre-allocate shared compressor buffers ADP does not use a shared compressor buffer pool; Rust request builders own fixed-capacity scratch and compression buffers.
entity_id Agent pod entity ID ADP internal DogStatsD telemetry uses OpenMetrics.
forwarder_requeue_buffer_size In-memory re-queue buffer size See below
heroku_dyno Heroku dyno telemetry mode See below
logging_frequency Transaction success log interval The core agent uses logging_frequency to throttle repetitive successful transaction logs. ADP logs successful forwarder operations below the default info level, so there is no matching info-level success-log stream to throttle. This key is intentionally unused.
otlp_config.metrics.batch.flush_timeout OTLP metrics batch flush timeout See below
otlp_config.metrics.batch.max_size Maximum OTLP metrics batch size See below
otlp_config.metrics.batch.min_size Minimum OTLP metrics batch size See below
telemetry.dogstatsd.aggregator_channel_latency_buckets Histogram buckets: DSD-to-aggregator channel lag ADP already tracks this latency with histogram buckets determined by logarithmic calculation, so pre-defined bucket configuration is not applicable.
telemetry.dogstatsd.listeners_channel_latency_buckets Histogram buckets: listener packet-channel latency ADP already tracks this latency with histogram buckets determined by logarithmic calculation, so pre-defined bucket configuration is not applicable.
telemetry.dogstatsd.listeners_latency_buckets Histogram buckets: listener processing latency ADP already tracks this latency with histogram buckets determined by logarithmic calculation, so pre-defined bucket configuration is not applicable.
use_dogstatsd Master DogStatsD enable toggle Core Agent evaluates and sets data_plane.dogstatsd.enabled.

aggregator_buffer_size

The core Agent uses aggregator_buffer_size to size bounded Go channels feeding the aggregator and DogStatsD time sampler workers.

ADP has no equivalent operator-facing setting. Its aggregate transform receives data through the Saluki topology interconnects, which are configured at topology construction time rather than by per-component Agent config keys. The default event interconnect capacity is 128 buffers, and each event buffer can hold up to 1024 events.

Setting aggregator_buffer_size has no effect in ADP.

aggregator_flush_metrics_and_serialize_in_parallel_buffer_size

The core Agent uses aggregator_flush_metrics_and_serialize_in_parallel_buffer_size during aggregator flushes, not for the aggregator's input queues. It controls how many flushed series or sketch objects are grouped into each internal buffered-channel slice while one goroutine produces flushed metrics and another goroutine serializes them.

ADP has no equivalent flush-and-serialize iterable pipeline. The aggregate transform emits aggregated Metric events into the Saluki topology through EventsBuffer batches, and downstream encoder components serialize those events independently of the aggregate transform.

The closest internal ADP buffering is topology-wide: EventsBuffer can hold up to 1024 events, and event interconnects default to 128 buffers. These are not configured through aggregator_flush_metrics_and_serialize_in_parallel_buffer_size, and setting this key has no effect in ADP.

aggregator_flush_metrics_and_serialize_in_parallel_chan_size

The core Agent uses aggregator_flush_metrics_and_serialize_in_parallel_chan_size during aggregator flushes, together with aggregator_flush_metrics_and_serialize_in_parallel_buffer_size. The buffer size controls how many flushed series or sketch objects are grouped into each internal slice, while the channel size controls how many of those slices can queue between the flush producer goroutine and the consumer goroutine that serializes them.

ADP has no equivalent flush-and-serialize iterable pipeline. The aggregate transform emits aggregated Metric events into the Saluki topology through EventsBuffer batches, and downstream encoder components serialize those events independently of the aggregate transform.

The closest internal ADP buffering is topology-wide: EventsBuffer can hold up to 1024 events, and event interconnects default to 128 buffers. These are not configured through aggregator_flush_metrics_and_serialize_in_parallel_chan_size, and setting this key has no effect in ADP.

aggregator_use_tags_store

The core Agent uses aggregator_use_tags_store to enable an aggregator-local, ref-counted tag store. The store deduplicates repeated tag slices across aggregator contexts: contexts retain shared tag entries while active, release them when they expire, and periodic shrinking removes entries that are no longer referenced.

ADP does not have an aggregator-local tag store and does not need this toggle. Metrics reach the aggregate transform with Saluki Context values that already carry ADP-native tag structures. Tag reuse is handled before aggregation by the context resolver, the tags resolver, and SharedTagSet structural sharing.

Related ADP-specific tuning is exposed through context/tag-set resolver settings such as dogstatsd_cached_tagsets_limit, not through aggregator_use_tags_store. Setting aggregator_use_tags_store has no effect in ADP.

data_plane.telemetry_enabled

Deprecated. ADP previously read this key to enable its internal Prometheus telemetry endpoint, but no longer does so. The key is silently ignored.

data_plane.telemetry_listen_addr

Deprecated. ADP previously read this key to configure the address of its internal Prometheus telemetry endpoint, but no longer does so. The key is silently ignored.

dogstatsd_mem_based_rate_limiter.enabled

The Core Agent exposes 11 keys under this prefix to apply backpressure when the Go process approaches its memory limit. They work by manipulating Go's garbage collector (debug.SetGCPercent, debug.FreeOSMemory), allocating a large heap ballast to adjust GC heuristics, and blocking goroutines to slow packet ingestion. None of these mechanisms have an equivalent in Rust, and ADP does not use a Go runtime.

ADP takes a different approach to the same problem using explicit static memory accounting and a process-level RSS limit. All 11 dogstatsd_mem_based_rate_limiter.* keys are not planned. See Memory Management for details.

DogStatsD statistics (dogstatsd_stats_enable / dogstatsd_metrics_stats_enable)

The core agent has two DogStatsD statistics mechanisms with different scopes. dogstatsd_stats_enable enables packet-level throughput statistics from a ring buffer, exposed as Go expvar data on dogstatsd_stats_port (default 5000). Operators must configure an OpenMetrics check to scrape that endpoint before the data is submitted. dogstatsd_metrics_stats_enable enables runtime-toggleable metric-level debug statistics that track count and last-seen time per unique metric and tag combination. That data powers the core agent's dogstatsd-stats CLI command and HTTP endpoint.

ADP does not mirror the packet-level statistics config path. Instead, ADP provides an on-demand metric-level view through a DogStatsD statistics destination that is always wired into the topology, but only collects data during a time-bounded request. To collect statistics, run agent-data-plane dogstatsd stats --duration-secs N or call the privileged /dogstatsd/stats?collection_duration_secs=N API. The handler waits for the requested collection window, then returns count and last-seen time per metric context inline as JSON. The CLI uses the same API and renders the result as either summary or cardinality analysis.

ADP also exposes aggregate DogStatsD counters through its internal telemetry scrape endpoint. This endpoint is separate from /dogstatsd/stats: it does not return the per-metric count and last-seen map, and it is not controlled by the core agent's dogstatsd_stats_* keys. See Scraping internal telemetry for endpoint details and scrape examples.

ADP does not expose the core agent's packet-per-second expvar endpoint or a persistent per-metric DogStatsD statistics endpoint to scrape. You do not need to set up scraper configuration for this per-metric data. The config keys dogstatsd_stats_enable, dogstatsd_stats_buffer, and dogstatsd_stats_port have no effect in ADP. See #1352.

forwarder_requeue_buffer_size

ADP does not implement forwarder_requeue_buffer_size. The core Agent uses this setting to size a separate count-bounded handoff channel from forwarder workers to the retry manager. ADP's endpoint I/O loop handles send failures that need to be retried and owns the retry queue, so ADP does not need a separate queue for worker-to-retry-manager communication. Those failures are re-enqueued into the low-priority retry queue, which is bounded by payload bytes via forwarder_retry_queue_payloads_max_size and optional disk persistence settings. See #1755.

heroku_dyno

The heroku_dyno setting affects the core Agent's self-telemetry heartbeat. It changes the Agent flavor used by the core Agent aggregator so the running heartbeat is emitted as datadog.heroku_agent.running.

ADP does not run in the supported Heroku Agent package path: the Heroku Agent package excludes the agent-data-plane dependency, and the Heroku Datadog launch script starts the core Agent, trace Agent, and optionally process Agent without launching an agent-data-plane process. ADP also does not emit the core Agent's datadog.<agentName>.running series.

Because the affected heartbeat is core-Agent-owned and ADP is not part of the supported Heroku deployment path, ADP does not implement heroku_dyno. See #1753.

otlp_config.metrics.batch.flush_timeout

The core Agent batches OTLP metric input before translation through its serializer exporter queue. ADP's architecture does not have the same batching mechanism and is unaffected by this configuration value.

otlp_config.metrics.batch.max_size

The core Agent batches OTLP metric input before translation through its serializer exporter queue. ADP's architecture does not have the same batching mechanism and is unaffected by this configuration value.

otlp_config.metrics.batch.min_size

The core Agent batches OTLP metric input before translation through its serializer exporter queue. ADP's architecture does not have the same batching mechanism and is unaffected by this configuration value.

Behavioral Differences

The following settings are recognized by both ADP and the core agent, but with different behavior or default values.

Config Key Description
aggregator_stop_timeout Timeout (s) for aggregator flush on stop
dogstatsd_mapper_cache_size Mapper result LRU cache size
dogstatsd_metrics_stats_enable Enable per-metric debug stats
forwarder_apikey_validation_interval API key check interval (minutes)
forwarder_high_prio_buffer_size High-priority request queue size
forwarder_num_workers Concurrent forwarder workers
forwarder_stop_timeout Timeout (s) for forwarder graceful stop
log_level Log verbosity directives
min_tls_version Minimum TLS version for HTTPS connections
multi_region_failover.enabled Enable multi-region failover mode
serializer_zstd_compressor_level Zstd compression level (Agent)
skip_ssl_validation Skip TLS cert validation
statsd_forward_host UDP packet forwarding destination host

aggregator_stop_timeout

The core Agent uses aggregator_stop_timeout as the shutdown grace period for the aggregator's final flush. During shutdown, the core Agent tries to flush aggregated metrics, events, service checks, and related data to the forwarder before stopping. If dogstatsd_flush_incomplete_buckets is enabled, the same timeout also bounds draining in-flight DogStatsD time sampler batches before that final flush.

ADP uses aggregator_stop_timeout together with forwarder_stop_timeout to configure its topology-wide graceful shutdown timeout. The default is 2 + 2 = 4 seconds. Set data_plane.stop_timeout to override the combined timeout directly.

Support is partial because ADP does not apply this timeout only to an aggregator component. Shutdown is coordinated by the Saluki topology: sources stop first, downstream inputs close, and the aggregate transform performs its final flush when its input stream ends. Whether open aggregation windows are included is controlled by aggregate_flush_open_windows, also aliased as dogstatsd_flush_incomplete_buckets.

dogstatsd_mapper_cache_size

ADP and the core agent both cache mapper results to skip regex evaluation on repeat metric names. With the default value of 1000, and with any positive integer, behavior matches the core agent: results are cached in an LRU keyed by the original metric name, including a negative-cache entry for names that match no profile.

The two implementations diverge when this setting is 0. In the core agent, 0 is rejected by the underlying LRU library, which causes the entire mapper to be silently disabled: mapping profiles configured by dogstatsd_mapper_profiles are not applied. In ADP, 0 disables the result cache only; mapping profiles still run, so each metric pays the regex evaluation cost without amortization.

If you previously set dogstatsd_mapper_cache_size: 0 in the core agent to turn off the mapper, clear dogstatsd_mapper_profiles instead when running ADP.

dogstatsd_metrics_stats_enable

See dogstatsd_stats_enable

forwarder_apikey_validation_interval

ADP supports forwarder_apikey_validation_interval for Datadog intake forwarding. The value is in minutes, defaults to 60, and falls back to that default for non-positive values.

ADP validates configured API keys when the forwarder starts, periodically at the configured interval, and after API-key-related runtime config updates. Validation controls readiness only: confirmed invalid keys can make the forwarder not ready, but the forwarder keeps running and transient validation failures do not prove that keys are invalid.

Support is partial because ADP only tracks additional_endpoints domains present at forwarder startup. Runtime updates can rotate keys for those domains, but brand-new domains are not added or validated until restart.

forwarder_high_prio_buffer_size

ADP implements forwarder_high_prio_buffer_size and matches the core agent's default of 100. The value sizes ADP's per-endpoint high-priority pending queue.

forwarder_num_workers

ADP uses forwarder_max_concurrent_requests to control endpoint concurrency. forwarder_num_workers is still read for HTTP connection pool sizing but no longer controls the maximum number of concurrent requests per endpoint.

forwarder_stop_timeout

The core Agent uses forwarder_stop_timeout as the shutdown grace period for the forwarder.

ADP uses forwarder_stop_timeout together with aggregator_stop_timeout to configure its topology-wide graceful shutdown timeout. The default is 2 + 2 = 4 seconds. Set data_plane.stop_timeout to override the combined timeout directly.

Support is partial because ADP does not apply this timeout only to a forwarder component. It bounds graceful shutdown for the full Saluki topology.

log_level

ADP accepts log_level as the startup logging control. A plain level applies to ADP-owned and Saluki-owned targets only, including agent_data_plane, saluki_*, and runtime crates under lib/.

log_level: debug

This keeps third-party dependencies such as hyper, tokio, and tonic at their default filtering unless you opt them in.

To control dependency logs or set a global fallback, use advanced EnvFilter directives in log_level. ADP applies those directive strings as configured:

log_level: warn,agent_data_plane=debug,hyper=warn

min_tls_version

ADP supports min_tls_version for Datadog intake forwarding through the shared Datadog forwarder. The default is tlsv1.2, which allows TLS 1.2 and TLS 1.3. To require TLS 1.3 only, set min_tls_version: tlsv1.3 or DD_MIN_TLS_VERSION=tlsv1.3.

The core agent also accepts tlsv1.0 and tlsv1.1. ADP accepts those values for configuration compatibility, but clamps them to TLS 1.2 because ADP uses rustls, which does not support TLS 1.0 or TLS 1.1.

This setting does not affect ADP IPC, local privileged APIs, ADP control-plane clients, OTLP proxying to the core agent, or unrelated HTTP clients.

multi_region_failover.enabled

ADP supports multi-region failover for metrics. When multi_region_failover.enabled and multi_region_failover.failover_metrics are both true, ADP forwards metrics to the primary Datadog endpoint and to a second MRF endpoint.

To enable the MRF metrics branch at startup, configure multi_region_failover.api_key and one of multi_region_failover.site or multi_region_failover.dd_url. If the MRF endpoint configuration is incomplete, ADP skips the MRF branch and continues forwarding to the primary endpoint.

ADP does not fall back to the primary api_key, site, or dd_url for MRF traffic. multi_region_failover.enabled and endpoint selection are resolved at startup. multi_region_failover.failover_metrics, multi_region_failover.metric_allowlist, and multi_region_failover.api_key can refresh from live configuration after the MRF branch is enabled.

Config Key Behavior Default
multi_region_failover.enabled Enables multi-region failover mode. false
multi_region_failover.failover_metrics Enables metrics forwarding to the failover region when MRF is enabled. false
multi_region_failover.metric_allowlist Exact metric names to forward to MRF. Empty or unset forwards all metrics. []
multi_region_failover.api_key API key for the failover-region endpoint. unset
multi_region_failover.site Datadog site for the failover region, used as https://app.mrf.<site>. unset
multi_region_failover.dd_url Explicit failover intake URL. Takes precedence over site when set. unset

serializer_zstd_compressor_level

The Core Agent's zstd compression level. ADP prefers data_plane.serializer_zstd_compressor_level; when that is unset, ADP applies this value if it has been changed from the Agent default of 1, otherwise it uses its own default of 3.

skip_ssl_validation

ADP supports skip_ssl_validation for Datadog intake forwarding through the shared Datadog forwarder. The default is false, which preserves normal server certificate validation. When enabled, this setting affects the Datadog intake clients used by metrics, logs, traces, events, and service checks that flow through the shared forwarder.

This setting does not affect ADP IPC, local privileged APIs, ADP control-plane clients, OTLP proxying to the core agent, or unrelated HTTP clients.

statsd_forward_host

ADP supports DogStatsD forwarding when both statsd_forward_host and statsd_forward_port are set. ADP forwards each framed DogStatsD message over UDP to the configured destination before parsing, filtering, mapping, or aggregation. Forwarding does not preserve the core Agent's packet-buffer grouping, so forwarded UDP datagrams may be split differently while carrying the same DogStatsD messages. ADP logs setup failures and tracks send failures through telemetry.

Compatibility Unknown

The following settings need further investigation. ADP behavior may differ from the core agent in ways that are not yet fully characterized.

Config Key Description Issue
forwarder_low_prio_buffer_size Low-priority request queue size #1362

ADP-Only Settings

The following settings are specific to ADP and have no equivalent in the core agent.

Config Key Description Default
aggregate_context_limit Max contexts per aggregation window
aggregate_flush_interval Aggregator flush period
aggregate_passthrough_idle_flush_timeout Passthrough buffer flush delay
aggregate_window_duration_seconds Aggregation window size
apm_config.obfuscation.sql.dbms SQL obfuscation DBMS dialect
apm_config.obfuscation.sql.dollar_quoted_func Preserve dollar-quoted SQL functions
apm_config.obfuscation.sql.keep_sql_alias Preserve SQL aliases in obfuscation
apm_config.obfuscation.sql.replace_digits Replace digits in SQL obfuscation
apm_config.obfuscation.sql.table_names Collect table names during obfuscation
counter_expiry_seconds Idle counter keep-alive duration 300
data_plane.otlp.receiver_grpc_endpoint_temporary ADP OTLP gRPC listen endpoint localhost:6317
data_plane.otlp.receiver_http_endpoint_temporary ADP OTLP HTTP listen endpoint localhost:6318
data_plane.serializer_zstd_compressor_level ADP zstd compression level 3
data_plane.stop_timeout ADP graceful shutdown timeout (s) derived
dogstatsd_allow_context_heap_allocs Allow heap allocations for contexts
dogstatsd_autoscale_udp_listeners Bind multiple UDP sockets via SO_REUSEPORT
dogstatsd_buffer_count_max Max receive buffers 256
dogstatsd_buffer_count Baseline receive buffers 128
dogstatsd_cached_contexts_limit Max cached metric contexts
dogstatsd_cached_tagsets_limit Max cached tagsets
dogstatsd_mapper_string_interner_size Mapper string interner capacity
dogstatsd_minimum_sample_rate Floor for metric sample rates
dogstatsd_permissive_decoding Relaxes decoder strictness true
dogstatsd_string_interner_size_bytes Explicit byte budget for context interner
dogstatsd_tcp_port TCP listen port for DSD
flush_timeout_secs Encoder flush timeout (secs)
logs_config.use_grpc Use stateful Foldspace transport for logs false
memory_limit Process memory limit
memory_slop_factor Memory accounting slop fraction 0.25
otlp_allow_context_heap_allocs Allow heap allocations for OTLP contexts
otlp_cached_contexts_limit Max cached OTLP metric contexts
otlp_cached_tagsets_limit Max cached OTLP tagsets
otlp_config.receiver.protocols.http.transport OTLP HTTP receiver transport
otlp_config.traces.enable_otlp_compute_top_level_by_span_kind Enable OTLP top-level-by-span-kind
otlp_config.traces.ignore_missing_datadog_fields Ignore missing Datadog fields in OTLP
otlp_config.traces.string_interner_size OTLP trace string interner capacity
otlp_string_interner_size OTLP context interner capacity
serializer_max_metrics_per_payload Max metrics per payload

logs_config.use_grpc

Enables the initial stateful logs transport. Failed or queued stateful payloads are reconstructed and stored as complete stateless HTTP transactions before entering the normal retry or persisted queues. This does not provide full crash durability: a process crash before conversion can lose unacknowledged stateful payloads, and the persisted queue retains its existing dequeue crash window.

data_plane.otlp.receiver_grpc_endpoint_temporary

Temporary development key for setting ADP's OTLP listen endpoints independently from the Agent's.

data_plane.otlp.receiver_http_endpoint_temporary

Temporary development key for setting ADP's OTLP listen endpoints independently from the Agent's.

data_plane.serializer_zstd_compressor_level

ADP-specific zstd compression level, taking precedence over the Core Agent's serializer_zstd_compressor_level. When this key is unset, ADP falls back to serializer_zstd_compressor_level if it has been changed from the Agent default of 1, and otherwise uses its own default of 3. Level 3 achieves ~6% smaller payloads (65.3 MB vs 69.3 MB) without a net CPU increase, since ADP is more efficient than the Agent and can afford higher compression. Configure via DD_DATA_PLANE_SERIALIZER_ZSTD_COMPRESSOR_LEVEL or in ADP-specific configuration.

data_plane.stop_timeout

ADP uses data_plane.stop_timeout as the topology-wide graceful shutdown timeout. If this key is unset, ADP defaults to aggregator_stop_timeout + forwarder_stop_timeout.

dogstatsd_minimum_sample_rate

ADP enforces a minimum sample rate on incoming metrics to prevent memory exhaustion from extremely low sample rates on histograms and sketches. Sending metrics with a very high inverse sample rate (for example @0.0000001) can cause unbounded memory growth in a sketch; this setting prevents that. The default is conservative enough that normal clients are unaffected.

dogstatsd_permissive_decoding

By default, ADP parses DogStatsD packets with the same leniency as the core agent, accepting packets that technically violate the spec. Setting this to false enables strict mode, which rejects non-conformant packets. Strict mode is not available in the core agent.

dogstatsd_string_interner_size_bytes

Accepts a bare integer number of bytes or a human-readable byte-size string such as 12MiB. When unset, ADP derives the byte budget from dogstatsd_string_interner_size.

memory_limit / memory_slop_factor

ADP uses an explicit process memory limit (memory_limit) rather than relying on Go's garbage collector. The memory_slop_factor reserves a fraction of the limit to account for allocations not tracked by ADP's internal accounting. When memory usage approaches memory_limit, ADP's global limiter begins exerting backpressure (see enable_global_limiter).

memory_slop_factor

See memory_limit above.

Transparent Settings

The following settings work in ADP with the same behavior as the core agent.

To enable syslog logging, set log_to_syslog: true. Console logging remains controlled by log_to_console; enabling syslog does not disable console or file logging. If syslog_uri is empty while syslog logging is enabled, ADP uses the platform default local syslog socket: unixgram:///dev/log on Linux and unixgram:///var/run/syslog on macOS. Set syslog_rfc: true when the receiving syslog daemon expects the Agent's RFC-style header.

UDS origin detection on macOS

ADP supports PID-based UDS origin detection on Linux. On macOS and other non-Linux platforms, Unix socket credentials do not expose the sender PID: LOCAL_PEERCRED and getpeereid() provide UID and GID only. ADP still accepts DogStatsD traffic over UDS on those platforms, but it cannot derive a process ID from the socket and therefore cannot use that path for origin enrichment.

When dogstatsd_origin_detection is enabled with a UDS listener on a non-Linux platform, ADP logs a startup warning and treats the missing PID as a static platform limitation rather than a per-packet origin detection error.

The on-demand PID resolver is also Linux-only because it reads procfs and cgroups. Containerd metadata can still populate ContainerPid aliases in the tag store on Unix platforms, which covers steady-state tagging when another source provides a PID, but macOS UDS origin detection cannot obtain that PID from the socket.

DogStatsD metric debug log

ADP supports the core agent's DogStatsD metric debug log. To write this file, set dogstatsd_metrics_stats_enable: true. dogstatsd_logging_enabled also must be true; it defaults to true, so most configurations only need to enable dogstatsd_metrics_stats_enable.

When dogstatsd_logging_enabled is true, ADP connects an extra DogStatsD destination to the decoded metric stream. The destination writes one line per metric sample with the metric name, tags, count, and last-seen time while dogstatsd_metrics_stats_enable is true. When dogstatsd_metrics_stats_enable is false, the destination drains decoded metrics and drops them. This lets runtime configuration changes start and stop the debug log without rebuilding the topology. This feature is for support and troubleshooting. It does not change normal metric forwarding, and it does not replace the on-demand /dogstatsd/stats API.

Use these settings to control the file:

Config Key Behavior
dogstatsd_log_file Output path. If empty, ADP uses the platform default DogStatsD stats log path.
dogstatsd_log_file_max_rolls Number of rotated files to keep. Defaults to 3.
dogstatsd_log_file_max_size Maximum active file size before rotation. Defaults to 10Mb.
dogstatsd_logging_enabled Controls whether ADP wires the debug log destination into the topology. Defaults to true.

The default dogstatsd_log_file path is /var/log/datadog/dogstatsd_info/dogstatsd-stats.log on Linux and other Unix platforms, /opt/datadog-agent/logs/dogstatsd_info/dogstatsd-stats.log on macOS, and %ProgramData%\datadog\logs\dogstatsd_info\dogstatsd-stats.log on Windows.

This debug log differs from the dogstatsd_capture_* settings. The debug log records decoded metric summaries after DogStatsD parsing. The capture settings record raw DogStatsD traffic for packet-level investigation, and they remain tracked separately under #1381.

Payload debug logging (log_payloads)

ADP supports log_payloads for debugging metric, event, and service check payload contents before they enter Datadog encoders. To see these logs, set log_payloads: true and run with debug-level logging enabled.

When enabled, ADP logs decoded payload objects: scalar series metrics, sketches/distributions, events, and service checks. These logs can contain high-volume customer data, including metric names, tags, host and container metadata, event text, and service check messages. Use this setting only while diagnosing payload content.

ADP does not dump the exact encoded JSON or protobuf HTTP request body, and it does not log compressed wire payload bytes.

Config Key Description
additional_endpoints Dual-ship to extra endpoints
agent_ipc.grpc_max_message_size Max inbound gRPC message size for IPC client
allow_arbitrary_tags Relax backend tag validation via HTTP header
api_key API key for endpoint auth
apm_config.compute_stats_by_span_kind Compute APM stats per span kind
apm_config.enable_rare_sampler Enable the APM rare-span sampler
apm_config.error_tracking_standalone.enabled Enable Error Tracking standalone
apm_config.errors_per_second APM error-span sampling rate (per sec)
apm_config.obfuscation.credit_cards.enabled apm_config.obfuscation.credit_cards.enabled
apm_config.obfuscation.credit_cards.keep_values apm_config.obfuscation.credit_cards.keep_values
apm_config.obfuscation.credit_cards.luhn apm_config.obfuscation.credit_cards.luhn
apm_config.obfuscation.elasticsearch.enabled apm_config.obfuscation.elasticsearch.enabled
apm_config.obfuscation.elasticsearch.keep_values apm_config.obfuscation.elasticsearch.keep_values
apm_config.obfuscation.elasticsearch.obfuscate_sql_values Obfuscate SQL values in ES queries
apm_config.obfuscation.http.remove_paths_with_digits Strip numeric segments from HTTP paths
apm_config.obfuscation.http.remove_query_string apm_config.obfuscation.http.remove_query_string
apm_config.obfuscation.memcached.enabled apm_config.obfuscation.memcached.enabled
apm_config.obfuscation.memcached.keep_command apm_config.obfuscation.memcached.keep_command
apm_config.obfuscation.mongodb.enabled apm_config.obfuscation.mongodb.enabled
apm_config.obfuscation.mongodb.keep_values apm_config.obfuscation.mongodb.keep_values
apm_config.obfuscation.mongodb.obfuscate_sql_values Obfuscate SQL values in MongoDB queries
apm_config.obfuscation.opensearch.enabled apm_config.obfuscation.opensearch.enabled
apm_config.obfuscation.opensearch.keep_values apm_config.obfuscation.opensearch.keep_values
apm_config.obfuscation.opensearch.obfuscate_sql_values Obfuscate SQL values in OpenSearch queries
apm_config.obfuscation.redis.enabled apm_config.obfuscation.redis.enabled
apm_config.obfuscation.redis.remove_all_args apm_config.obfuscation.redis.remove_all_args
apm_config.obfuscation.valkey.enabled apm_config.obfuscation.valkey.enabled
apm_config.obfuscation.valkey.remove_all_args apm_config.obfuscation.valkey.remove_all_args
apm_config.peer_tags Extra peer tags for stats aggregation
apm_config.peer_tags_aggregation Aggregate APM stats by peer tags
apm_config.probabilistic_sampler.enabled Enable APM probabilistic sampler
apm_config.probabilistic_sampler.sampling_percentage Probabilistic sampler percentage
apm_config.target_traces_per_second Target sampled traces per second
autoscaling.failover.enabled Enable autoscaling failover metric routing
autoscaling.failover.metrics Metric names forwarded to DCA for failover
basic_telemetry_add_container_tags Add container tags to basic telemetry signals
bind_host Global listen host fallback
cluster_agent.auth_token Bearer token for Cluster Agent requests
cluster_agent.enabled Enable Cluster Agent communication
cluster_agent.kubernetes_service_name Cluster Agent Kubernetes service name
cluster_agent.url Cluster Agent HTTPS endpoint
cmd_port Core Agent CMD API port for ADP gRPC IPC
cri_connection_timeout CRI container runtime connection timeout (s)
cri_query_timeout CRI container runtime query timeout (s)
data_plane.api_listen_address Unprivileged API listen address
data_plane.dogstatsd.aggregator_tag_filter_cache_capacity Tag-filter deduplication cache size
data_plane.dogstatsd.enabled Enable the DogStatsD pipeline
data_plane.enabled Enable the data plane
data_plane.log_file ADP log file path
data_plane.otlp.enabled Enable the OTLP pipeline
data_plane.otlp.proxy.enabled Enable OTLP proxy to Core Agent
data_plane.otlp.proxy.logs.enabled Proxy OTLP logs to Core Agent
data_plane.otlp.proxy.metrics.enabled Proxy OTLP metrics to Core Agent
data_plane.otlp.proxy.receiver.protocols.grpc.endpoint OTLP proxy gRPC receiver endpoint
data_plane.otlp.proxy.traces.enabled Proxy OTLP traces to Core Agent
data_plane.remote_agent_enabled Enable remote agent mode
data_plane.secure_api_listen_address Privileged API listen address
data_plane.use_new_config_stream_endpoint Use new config stream endpoint
dd_url Override intake endpoint URL
disable_file_logging Disable writing logs to a file
dogstatsd_buffer_size Receive buffer size (bytes)
dogstatsd_capture_depth Traffic capture channel depth
dogstatsd_capture_path Traffic capture file location
dogstatsd_context_expiry_seconds Context cache TTL (seconds)
dogstatsd_disable_verbose_logs Suppress noisy parse error logs
dogstatsd_entity_id_precedence Entity ID over auto-detection
dogstatsd_eol_required Require newline-terminated messages
dogstatsd_expiry_seconds Counter value expiry (seconds)
dogstatsd_flush_incomplete_buckets Flush open buckets on shutdown
dogstatsd_log_file DSD dedicated log file path
dogstatsd_log_file_max_rolls DSD log file max roll count
dogstatsd_log_file_max_size DSD log file max size
dogstatsd_logging_enabled Enables DSD metric logging
dogstatsd_mapper_profiles Metric mapping profile definitions
dogstatsd_no_aggregation_pipeline Enable no-agg timestamped path
dogstatsd_non_local_traffic Accept non-localhost UDP/TCP
dogstatsd_origin_detection Enable UDS origin detection
dogstatsd_origin_detection_client Honor client origin proto fields
dogstatsd_origin_optout_enabled Allow clients to opt out origin
dogstatsd_pipe_name Windows named pipe path
dogstatsd_port UDP listen port
dogstatsd_so_rcvbuf Socket receive buffer size
dogstatsd_socket UDS datagram socket path
dogstatsd_stream_log_too_big Log oversized UDS stream frames
dogstatsd_stream_socket UDS stream socket path
dogstatsd_string_interner_size String interner capacity
dogstatsd_tag_cardinality Default tag cardinality level
dogstatsd_tags Extra tags added to all DSD data
dogstatsd_windows_pipe_security_descriptor Windows named pipe ACL descriptor
enable_payloads.events Allow sending event payloads
enable_payloads.series Allow sending series payloads
enable_payloads.service_checks Allow sending service check payloads
enable_payloads.sketches Allow sending sketch payloads
env Agent environment name
expected_tags_duration How long startup host tags are attached
forwarder_backoff_base Retry backoff base (secs)
forwarder_backoff_factor Retry backoff jitter factor
forwarder_backoff_max Retry backoff ceiling (secs)
forwarder_connection_reset_interval HTTP conn reset interval (secs)
forwarder_flush_to_disk_mem_ratio Mem-to-disk flush ratio
forwarder_http_protocol HTTP version selection (auto/http1/http2)
forwarder_max_concurrent_requests Max concurrent HTTP requests
forwarder_outdated_file_in_days Days before retry files are deleted
forwarder_recovery_interval Backoff recovery decrease factor
forwarder_recovery_reset Reset errors on success
forwarder_retry_queue_capacity_time_interval_sec Retry queue time-based capacity
forwarder_retry_queue_max_size Retry queue max size (deprecated)
forwarder_retry_queue_payloads_max_size Retry queue max size (bytes)
forwarder_storage_max_disk_ratio Max disk usage ratio for retry
forwarder_storage_max_size_in_bytes Max on-disk retry storage size
forwarder_storage_path On-disk retry storage directory
forwarder_timeout Forwarder HTTP request timeout
histogram_aggregates Histogram aggregate statistics
histogram_copy_to_distribution Copy histograms to distributions
histogram_copy_to_distribution_prefix Prefix for hist-to-dist copies
histogram_percentiles Histogram percentile aggregates
log_file_max_rolls Max rolled log files to retain
log_file_max_size Max log file size before rolling
log_format_json Emit logs as JSON
log_format_rfc3339 Use RFC 3339 timestamps in log output
log_payloads Debug-log decoded payload contents before encoding
log_to_console Write logs to the console
log_to_syslog Write logs to syslog
metric_filterlist Metric name blocklist
metric_filterlist_match_prefix Blocklist uses prefix matching
metric_tag_filterlist Per-metric tag allow/deny filter list
multi_region_failover.api_key API key for the failover-region endpoint
multi_region_failover.dd_url Failover intake URL
multi_region_failover.failover_metrics Enable metrics forwarding to failover region
multi_region_failover.metric_allowlist Metric name allowlist for MRF forwarding
multi_region_failover.site Datadog site for the failover region
no_proxy_nonexact_match Domain/CIDR no_proxy matching
observability_pipelines_worker.metrics.enabled Route metrics to OPW instance
observability_pipelines_worker.metrics.url OPW metrics intake URL
observability_pipelines_worker.metrics.use_v3_api.series Use V3 series for OPW
origin_detection_unified Unified origin detection mode
otlp_config.logs.enabled otlp_config.logs.enabled
otlp_config.metrics.enabled otlp_config.metrics.enabled
otlp_config.metrics.histograms.mode OTLP histogram bucket reporting mode
otlp_config.metrics.histograms.send_aggregation_metrics Emit OTLP histogram aggregation metrics.
otlp_config.metrics.resource_attributes_as_tags Add scalar resource attributes as raw tags.
otlp_config.metrics.summaries.mode OTLP summary quantile reporting mode.
otlp_config.metrics.sums.cumulative_monotonic_mode Cumulative monotonic sum reporting mode.
otlp_config.metrics.sums.initial_cumulative_monotonic_value Initial cumulative sum reporting behavior.
otlp_config.metrics.tags Comma-separated tags for all OTLP metrics.
otlp_config.receiver.protocols.grpc.endpoint otlp_config.receiver.protocols.grpc.endpoint
otlp_config.receiver.protocols.grpc.max_recv_msg_size_mib Max OTLP inbound gRPC message size (MiB)
otlp_config.receiver.protocols.grpc.transport otlp_config.receiver.protocols.grpc.transport
otlp_config.receiver.protocols.http.endpoint otlp_config.receiver.protocols.http.endpoint
otlp_config.traces.enabled otlp_config.traces.enabled
otlp_config.traces.internal_port otlp_config.traces.internal_port
otlp_config.traces.probabilistic_sampler.sampling_percentage OTLP trace sampling percentage
provider_kind Provider kind static tag
proxy.http HTTP proxy URL
proxy.https HTTPS proxy URL
proxy.no_proxy Hosts bypassing proxy
serializer_compressor_kind Payload compression algorithm
serializer_experimental_use_v3_api.compression_level V3 API zstd compression level
serializer_experimental_use_v3_api.series.beta_route Beta V3 series API route
serializer_experimental_use_v3_api.series.endpoints Endpoints enabled for V3 series API
serializer_experimental_use_v3_api.series.shadow_sample_rate V3 series shadow mode sample rate
serializer_experimental_use_v3_api.series.shadow_sites V3 series shadow mode enabled sites
serializer_experimental_use_v3_api.series.use_beta Use the beta V3 series API route
serializer_experimental_use_v3_api.series.validate Dual-send v2+v3 series for validation
serializer_experimental_use_v3_api.sketches.endpoints Endpoints enabling v3 sketches API
serializer_experimental_use_v3_api.sketches.validate Dual-send v2+v3 sketches for validation
serializer_max_payload_size Max compressed payload size (generic)
serializer_max_series_payload_size Max compressed V2 series payload size
serializer_max_series_points_per_payload Max data points per series payload
serializer_max_series_uncompressed_payload_size Max uncompressed V2 series payload size
serializer_max_uncompressed_payload_size Max uncompressed payload size (generic)
site Datadog site domain
sslkeylogfile Write TLS session keys to file
statsd_forward_port UDP packet forwarding destination port
statsd_metric_blocklist List of metric names to block/drop
statsd_metric_blocklist_match_prefix Treat blocklist entries as prefixes
statsd_metric_namespace Prefix prepended to all metric names
statsd_metric_namespace_blacklist Prefixes exempt from namespace
syslog_rfc Use RFC-style syslog header
syslog_uri Syslog destination URI
telemetry.dogstatsd_origin Per-origin processed-metrics telemetry
use_proxy_for_cloud_metadata Proxy cloud metadata endpoints
use_v2_api.series Send series via V2 protobuf endpoint
use_v3_api.series.enabled Global V3 series mode
use_v3_api.series.endpoints Per-endpoint V3 series modes
vector.metrics.enabled Route metrics to OPW (legacy alias)
vector.metrics.url OPW metrics intake URL (legacy alias)
vector.metrics.use_v3_api.series Use V3 series for Vector
vsock_addr vsock address for Agent IPC endpoint