Skip to content

Commit b385a84

Browse files
authored
Collect Discovery Probe Telemetry (#53270)
### What does this PR do? Forwards telemetry for new discovery probe feature in the Agent. ### Motivation We are planning on probing services to determine the relevant ports that are exposing telemetry. We want to know the success and failure rate of this new feature in order to guide the iterative development of the probes. ### Additional Notes Overall cardinality of this data should be relatively low. `discovery_queue_depth` and `discovery_results` have tag context of `entity_kind` and `integration_name`. Generally speaking, an Agent will only either be monitoring `entity_kind: container` or `entity_kind: process` so the cardinality multiplier here is just `x1`. The `integration_name` will be the set of services on a host that match the probing profile. For instance if a host has 1000 unique services running but only 10 are eligible candidates for probing, then there will only be 10 unique metric contexts created _once_. This data is only emitted once during service startup. The eligible candidates for probing will be a short list of less than 100 specific open source processes like redis, krakendb, couchbase, calico, memcache, kafka, etc. Co-authored-by: gabe.dossantos <gabe.dossantos@datadoghq.com>
1 parent b66547a commit b385a84

2 files changed

Lines changed: 20 additions & 3 deletions

File tree

comp/core/agenttelemetry/impl/agenttelemetry_test.go

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -616,9 +616,8 @@ func TestRun(t *testing.T) {
616616
totalProfiles += len(job.profiles)
617617
}
618618
fmt.Println(totalProfiles)
619-
620-
// Default config has 18 profiles total (checks, logs-and-metrics, database, synthetics, connectivity, csi-driver, service-discovery, runtime-started, runtime-running, hostname, rtloader, otlp, procmgr, trace-agent, gpu, cluster-agent, injector, ebpf)
621-
assert.Equal(t, 18, totalProfiles)
619+
// Default config has 19 profiles total (checks, logs-and-metrics, database, synthetics, connectivity, csi-driver, service-discovery, runtime-started, runtime-running, hostname, rtloader, otlp, procmgr, trace-agent, gpu, cluster-agent, injector, ebpf, autodiscovery-discovery-probe)
620+
assert.Equal(t, 19, totalProfiles)
622621
}
623622

624623
func TestReportMetricBasic(t *testing.T) {

comp/core/agenttelemetry/impl/defaultProfiles.yaml

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -453,6 +453,24 @@ profiles:
453453
start_after: 30
454454
iterations: 0
455455
period: 900
456+
- name: autodiscovery-discovery-probe
457+
metric:
458+
exclude:
459+
zero_metric: true
460+
metrics:
461+
- name: autodiscovery.discovery_queue_depth
462+
preserve_tags:
463+
- integration_name
464+
- entity_kind
465+
- name: autodiscovery.discovery_results
466+
preserve_tags:
467+
- integration_name
468+
- result
469+
- entity_kind
470+
schedule:
471+
start_after: 30
472+
iterations: 0
473+
period: 900
456474
- name: ebpf
457475
metric:
458476
exclude:

0 commit comments

Comments
 (0)