Skip to content

Commit 85a3c43

Browse files
authored
chore(telemetry): fix known-metrics generator and regenerate from ddoghq/dd-go (#5191)
### What does this PR do? Three commits, each independently reviewable: 1. **`9a33e516c` — regenerate the known-metrics allowlists.** Adds 36 metric declarations that dd-go had accumulated since the last refresh in May 2026. 2. **`ebaf134d1` — repoint the generator at `ddoghq/dd-go`.** dd-go moved orgs; the generator still used the old `DataDog/dd-go` path. 3. **`91d34ef05` — run the generated source through `go/format`.** Makes regeneration idempotent. ### Motivation Follow-up to review feedback on #5032 (merged), which submits `telemetry.Count(telemetry.NamespaceTracers, "stats_collapsed_spans", ...)` when client-side stats hit a cardinality limit. Every `telemetry.Count` handle is validated against `internal/telemetry/internal/knownmetrics`, and `metrics.LoadOrStore` logs a warning on first use of an unknown name — so applications hitting a cardinality limit emit avoidable telemetry warnings today, since the emission is already on `main` without the allowlist entry. **No dd-go change is needed.** `stats_collapsed_spans` already exists in `trace/apps/tracer-telemetry-intake/telemetry-metrics/static/common_metrics.json` under `tracers` with `metric_type: count` and tags `[collapsed, oversized]`, added in dd-go `b60cc2775a6a` and merged to `prod`. The dd-trace-go generated files were simply stale. Chasing that down surfaced two problems with the generator itself, hence commits 2 and 3. ### The generator pointed at a stale mirror dd-go now lives at `github.com/ddoghq/dd-go`, but the generator's base64-encoded URLs still referenced `DataDog/dd-go`. That path does **not** redirect — the GitHub API returns `200` with `full_name: DataDog/dd-go`, `archived: false`, serving a copy whose `prod` stopped tracking upstream around 2026-08-04. So regeneration silently read week-old data. Repointing picked up **23 further metrics** that were already in dd-go `prod` but invisible from the old path — all `civisibility`: `ddtest.cli.*`, `ddtest.planning.*`, `ddtest.test_discovery.*`, `test_suite_durations.*`. The new path is private to the `ddoghq` org, so an unauthorized token now fails with an explicit message rather than a bare `unexpected status code: 404 Not Found`. Worth noting for contributors: the `gh auth token` fallback uses whichever account is *active*, so anyone with multiple GitHub accounts needs `gh auth token -u <ddoghq-account>`. Someone with dd-go admin access may want to archive `DataDog/dd-go` — a live repo serving a stale `prod` will keep catching people out. ### The generator emitted unformatted Go `template.tmpl` is not gofmt-clean (two-space indents, `{ Type: ... }` spacing) and nothing formatted its output, so every run rewrote both files wholesale. The first regeneration here produced 274 insertions / 238 deletions of whitespace churn around 36 real additions. Commit 3 renders into a buffer, runs `format.Source`, then writes. Regenerating on top of the formatted files is now byte-identical, so `make generate` is idempotent and future diffs show only real metric changes. ### How to regenerate ```bash GITHUB_TOKEN=$(gh auth token -u <ddoghq-account>) go run ./internal/telemetry/internal/knownmetrics/generator ``` ### Testing `go vet ./internal/telemetry/internal/knownmetrics/...` and `go build ./internal/telemetry/...` pass. `gofmt -l` is clean, and a second generator run against the committed state produces an empty diff. There are no test files in this package; the generated data is validated by CI's generate check. ### Reviewer's Checklist - [ ] Changed code has unit tests for its functionality at or near 100% coverage. - [ ] [System-Tests](https://github.com/DataDog/system-tests/) covering this feature have been added and enabled with the va.b.c-dev version tag. - [ ] There is a benchmark for any new code, or changes to existing code. - [ ] If this interacts with the agent in a new way, a system test has been added. - [x] New code is free of linting errors. You can check this by running `make lint` locally. - [x] New code doesn't break existing tests. You can check this by running `make test` locally. - [ ] Add an appropriate team label so this PR gets put in the right place for the release notes. - [x] All generated files are up to date. You can check this by running `make generate` locally. - [ ] Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: sam.maya <sam.maya@datadoghq.com>
1 parent 9284908 commit 85a3c43

3 files changed

Lines changed: 81 additions & 10 deletions

File tree

internal/telemetry/internal/knownmetrics/generator/generator.go

Lines changed: 22 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import (
1212
"encoding/json"
1313
"flag"
1414
"fmt"
15+
"go/format"
1516
"net/http"
1617
"net/url"
1718
"os"
@@ -30,8 +31,8 @@ import (
3031
// This represents the base64-encoded URL of api.github.com to download the configuration file.
3132
// This can be easily decoded manually, but it is encoded to prevent the URL from being scanned by bots.
3233
const (
33-
commonMetricsURL = "aHR0cHM6Ly9hcGkuZ2l0aHViLmNvbS9yZXBvcy9EYXRhRG9nL2RkLWdvL2NvbnRlbnRzL3RyYWNlL2FwcHMvdHJhY2VyLXRlbGVtZXRyeS1pbnRha2UvdGVsZW1ldHJ5LW1ldHJpY3Mvc3RhdGljL2NvbW1vbl9tZXRyaWNzLmpzb24="
34-
goMetricsURL = "aHR0cHM6Ly9hcGkuZ2l0aHViLmNvbS9yZXBvcy9EYXRhRG9nL2RkLWdvL2NvbnRlbnRzL3RyYWNlL2FwcHMvdHJhY2VyLXRlbGVtZXRyeS1pbnRha2UvdGVsZW1ldHJ5LW1ldHJpY3Mvc3RhdGljL2dvbGFuZ19tZXRyaWNzLmpzb24="
34+
commonMetricsURL = "aHR0cHM6Ly9hcGkuZ2l0aHViLmNvbS9yZXBvcy9kZG9naHEvZGQtZ28vY29udGVudHMvdHJhY2UvYXBwcy90cmFjZXItdGVsZW1ldHJ5LWludGFrZS90ZWxlbWV0cnktbWV0cmljcy9zdGF0aWMvY29tbW9uX21ldHJpY3MuanNvbg=="
35+
goMetricsURL = "aHR0cHM6Ly9hcGkuZ2l0aHViLmNvbS9yZXBvcy9kZG9naHEvZGQtZ28vY29udGVudHMvdHJhY2UvYXBwcy90cmFjZXItdGVsZW1ldHJ5LWludGFrZS90ZWxlbWV0cnktbWV0cmljcy9zdGF0aWMvZ29sYW5nX21ldHJpY3MuanNvbg=="
3536
)
3637

3738
//go:embed template.tmpl
@@ -62,6 +63,10 @@ func downloadFromDdgo(remoteURL, localPath, branch, token string, getMetricNames
6263

6364
defer response.Body.Close()
6465

66+
if response.StatusCode == http.StatusNotFound {
67+
return fmt.Errorf("%s: %s (the dd-go repository is private to the ddoghq organization; make sure the token you provide is authorized for it)", response.Status, remoteURL)
68+
}
69+
6570
if response.StatusCode != http.StatusOK {
6671
return fmt.Errorf("unexpected status code: %s", response.Status)
6772
}
@@ -83,17 +88,24 @@ func downloadFromDdgo(remoteURL, localPath, branch, token string, getMetricNames
8388
return strings.Compare(i.Name, j.Name)
8489
})
8590

86-
fp, err := os.Create(localPath)
87-
if err != nil {
88-
return err
89-
}
90-
defer fp.Close()
91-
91+
var rendered bytes.Buffer
9292
codegen := template.Must(template.New("").Parse(codegenTemplate))
93-
return codegen.Execute(fp, map[string]any{
93+
if err := codegen.Execute(&rendered, map[string]any{
9494
"symbolName": symbolName,
9595
"metrics": metricNames,
96-
})
96+
}); err != nil {
97+
return err
98+
}
99+
100+
// The template is not gofmt-clean on its own, so format the rendered source
101+
// before writing it out. Otherwise every run rewrites the whole file and the
102+
// real changes are lost in a sea of whitespace noise.
103+
formatted, err := format.Source(rendered.Bytes())
104+
if err != nil {
105+
return fmt.Errorf("formatting generated source for %s: %w", localPath, err)
106+
}
107+
108+
return os.WriteFile(localPath, formatted, 0644)
97109
}
98110

99111
func getCommonMetricNames(input map[string]any) []knownmetrics.Declaration {

internal/telemetry/internal/knownmetrics/known_metric.golang.go

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

internal/telemetry/internal/knownmetrics/known_metrics.common.go

Lines changed: 57 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)