Skip to content

feat(metrics): add stateful Foldspace transport - #2318

Draft
aqian01 wants to merge 1 commit into
mainfrom
andrewq/foldspace-stateful-metrics
Draft

feat(metrics): add stateful Foldspace transport#2318
aqian01 wants to merge 1 commit into
mainfrom
andrewq/foldspace-stateful-metrics

Conversation

@aqian01

@aqian01 aqian01 commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • route authoritative V3 metric series as versioned, compressed logical batches
  • add independent Foldspace gRPC workers per endpoint/API key with bounded ordered in-flight state
  • retain and retry complete logical metrics in memory or on disk, encoding only when gRPC has usable capacity
  • keep sketches and unit-bearing series on their existing HTTP paths

Motivation

Stateful metric encoding must happen after endpoint routing so each destination owns independent dictionary and stream state. The complete logical batch remains retained until ACK, which allows an unacknowledged batch to be encoded again against confirmed state after a transport failure or reconnect.

A failure after transmission has unknown delivery, so a resent logical batch may be duplicated, but its metric fields and identity are preserved.

Retry behavior

  • no HTTP fallback for stateful metric retries
  • no retry dequeue or encoding while the stream is disconnected or lacks send capacity
  • reconnect backoff with jitter
  • fresh-data priority with bounded progress for memory and disk retries
  • explicit memory/disk limits, drop telemetry, and graceful-shutdown persistence

Configuration

  • serializer_experimental_use_v3_api.stateful_metrics_enabled
  • stateful_metrics_max_inflight_payloads (default: 32)

Dependency

Depends on DataDog/foldspace#39 and pins Foldspace commit 7a5feff72311a66e8b6ce004f78b578f733a2b8f.

Validation

  • cargo test -p saluki-components common::datadog::stateful_metrics — 6 passed
  • cargo test -p saluki-components encoders::datadog::metrics — 71 passed
  • cargo test -p saluki-components common::datadog::config — 34 passed
  • cargo test -p saluki-io net::util::retry — 34 passed
  • cargo clippy -p saluki-components -p saluki-core -p saluki-io --all-targets -- -D warnings
  • cargo +nightly fmt --all -- --check
  • git diff --check

@dd-octo-sts dd-octo-sts Bot added area/core Core functionality, event model, etc. area/io General I/O and networking. area/components Sources, transforms, and destinations. area/docs Reference documentation. encoder/datadog-metrics Datadog Metrics encoder. forwarder/datadog Datadog forwarder. labels Aug 12, 2026
@pr-commenter

pr-commenter Bot commented Aug 12, 2026

Copy link
Copy Markdown

Regression Detector (Agent Data Plane)

Optimization Goals: ⚠️ Report unavailable

The benchmark run did not produce a usable report: outputs/report.json is missing

Check the benchmark job logs for details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/components Sources, transforms, and destinations. area/core Core functionality, event model, etc. area/docs Reference documentation. area/io General I/O and networking. encoder/datadog-metrics Datadog Metrics encoder. forwarder/datadog Datadog forwarder.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant