Define dataflows for agent-integrations-owned integrations - #3101
Define dataflows for agent-integrations-owned integrations#3101philjlee wants to merge 1 commit into
Conversation
Split of #3093 by CODEOWNERS team. Jira: TXP-277
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aa82e7c512
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -0,0 +1,6 @@ | |||
| provides: | |||
| - id: lacework-logs | |||
There was a problem hiding this comment.
Declare Lacework's event dataflow
When a customer selects Events Summary, Lacework sends summaries to the Datadog Events platform, as documented in lacework/README.md lines 20-24 and 46-48. Advertising only lacework-logs therefore causes dataflow consumers to omit a supported ingestion path; add a corresponding inbound lacework-events entry.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
I can only run on private repositories.
| @@ -0,0 +1,6 @@ | |||
| provides: | |||
| - id: redis-sentinel-metrics | |||
There was a problem hiding this comment.
Declare Redis Sentinel's failover event dataflow
When a monitored master's IP changes after initialization, _process_master_stats calls self.event(...) for the failover (redis_sentinel.py lines 219-233). Declaring only redis-sentinel-metrics therefore hides a data type that the installed check actually emits; add a corresponding inbound redis-sentinel-events entry.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
I can only run on private repositories.
Jira: TXP-277
Adds
assets/dataflows.yamlto 9 integrations owned by@DataDog/agent-integrations, declaring 11 dataflows. Mechanical change: no code, no metrics, no manifests touched.Precedent: #2925 "Define dataflows for saas-integrations", which created 13 of the 14 existing dataflow files in this repo.
Selection criteria
A directory is in this batch only if all of the following hold:
assets/dataflows.yamltoday.manifest.json. The dataflows validator hard-requires one —dataflows_validation_handler.go:56.data_typeis mechanically derivable from a committed artifact, with no judgement call:metadata.csvwith at least one data row →metricsassets/logs/*.yaml→logs.github/CODEOWNERSresolves<dir>/assets/dataflows.yaml(last-match-wins) to@DataDog/agent-integrations.Every directory in this batch has
@DataDog/agent-integrationsas its Datadog team CODEOWNER (mendixis co-owned with@DataDog/ecosystems-review), so review from@DataDog/agent-integrationscovers the whole PR.Field values
always_on: true/granular: false/direction: inboundmatches 14 of the 17 dataflow entries already in the repo. The only deviation in the repo isvercel, which isalways_on: falsefor a Serverless product-enablement reason that does not apply here.Dataflow IDs are
<app_id>-<data_type>, takingapp_idfrommanifest.jsonrather than the directory name. All IDs were checked for collisions against every existing ID in the repo, and against each other.Validation
These files were validated by executing the real validator,
DataflowsValidationHandlerfromdd-source/domains/integrationscatalog/libs/catalogassetslib/dataflows_validation_handler.go, atddoghq/dd-source@main, against everydataflows.yamlfile in the repo working tree with this PR applied. Result: 0 failures, covering per-file unmarshalling, proto constraint validation, and the cross-fileHandleLibraryID-uniqueness check.The harness was negative-tested first and confirmed to reject: a missing
always_on; adata_typeoutsidevalidDataTypes; anidbreaking^[a-z0-9-]+$; anidunder 3 characters; a file with neitherprovidesnoruses; a.ymlextension; a missingmanifest.json; and the same dataflow ID provided by two apps.This matters because APW does not post validator comments on integrations-extras PRs (
enable_validator_commentsis set only forpub-platform-stagingandpublishing-platform). A malformeddataflows.yamlmerges cleanly here and only fails afterwards, in the shared asset pipeline.No
data_typewas guessed anywhere. Any integration whose correct value was not unambiguous is deferred to a later batch rather than approximated.Contents
Metrics only (5)
concourse_ciconcourse-ci-metricslighthouselighthouse-metricsmendixmendix-metricsredis_sentinelredis-sentinel-metricsvns3vns3-metricsLogs only (2)
laceworklacework-logssqreensqreen-logsMetrics and logs (2)
auth0auth0-metrics,auth0-logsbind9bind9-metrics,bind9-logs🤖 Generated with Claude Code