perf(resolver): avoid per-query label map allocations in metrics resolver

[0xERR0R]

Follow-up to the review of #2222, implementing the points raised there.

Important

Stacked on #2222, which is not merged yet, so the diff against main currently
also contains that PR's two commits. Only 05cd2a8 and b229157 belong to this PR.
Rebase onto main once #2222 lands.

1. WithLabelValues instead of With(prometheus.Labels{…})

Every counter in MetricsResolver.Resolve built a label map, on the path taken by every
DNS query. #2222 added a third one. WithLabelValues passes the values positionally and
allocates nothing.

Benchmarked over MetricsResolver.Resolve (RESOLVED response, one client name):

	before	after
GOGC default	4593 ns/op, 1008 B/op, 6 allocs/op	489 ns/op, 0 B/op, 0 allocs/op
GOGC=off	1245 ns/op, 1008 B/op, 6 allocs/op	480 ns/op, 0 B/op, 0 allocs/op

The direct cost of the map literals is ~765 ns/query; the rest of the gap is GC pressure
from ~1 KB of garbage per query. Real-world gain sits between the two numbers, depending
on the overall allocation load. The benchmark was not committed — it needs a stub resolver
that does not belong in the package.

2. Document what response_type actually contains

blocky_client_response_total was justified by response_type being the bounded 12-value
enum. The exposed set differs from it in both directions:

• err is a 13th value. responseType starts as "err" and is only overwritten when
  the chain returned a response, so failures land in the counter. An alert written as
  response_type!~"BLOCKED" — the shape proposed in blocky_query_total can't be split by blocklist reason — no way to alert on "real" vs blocked query rate #2198 — silently counts failures as
  successfully resolved traffic. Now named as a constant, stated in the metric's Help
  text and in the docs.
• FILTERED and NOTFQDN can never appear. FilteringResolver and FQDNOnlyResolver
  answer those queries themselves and sit above MetricsResolver in the chain, so the
  FILTERED clause in the proposed alert is dead. They are missing from blocky_query_total,
  blocky_response_total and blocky_request_duration_seconds too — pre-existing, but worth
  writing down. See the note at the end.

Also documented: the counter increments once per query, so it sums to blocky_query_total,
not to the similarly named blocky_response_total, which skips the error path.

3. Give the cardinality note a lever

#2222 correctly warns that client is unbounded, but config.Metrics only has enable
and path, so the note left readers with nothing to do. It now shows the Prometheus-side
mitigation — and warns that the obvious one is wrong: dropping only the client label
collapses the per-client series into one and makes Prometheus reject the scrape with a
duplicate-sample error. Dropping the two metrics is the working option.

4. Tests

• blocky_client_response_total for a BLOCKED response — the metric's whole purpose,
  previously untested.
• multi-name clients joining into name1,name2, covering the clientLabel value now
  shared between two counters.
• metrics disabled: nothing is recorded.
• e2e: asserts the counter reaches /metrics with a real client label and both
  response_type="BLOCKED" and response_type="RESOLVED". This is what pins the positional
  WithLabelValues order end-to-end.

Each new spec was mutation-checked (break the expectation → spec fails), so none pass vacuously.

Not included

• Grafana dashboard. blocky-grafana.json has a topk(10, sum by (client) (…blocky_query_total…))
  panel; a "top blocked clients" companion would fit, but that is a dashboard change with its
  own review surface.
• A config toggle for the client label. You mentioned planning this in blocky_query_total can't be split by blocklist reason — no way to alert on "real" vs blocked query rate #2198 — it is the
  real fix for the cardinality note and deserves its own PR.
• Moving MetricsResolver above filtering/fqdnOnly so FILTERED/NOTFQDN become
  visible. Documented here rather than changed, since it shifts existing metric semantics.
  Happy to open an issue for it.

Verification

• go build ./...
• go test -race ./resolver/... ./metrics/... — pass
• go test ./resolver/... ./metrics/... ./server/... ./stats/... — pass
• make lint (golangci-lint v2.12.2) — 0 issues
• e2e metrics spec against a locally built blocky-e2e image — pass

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.05%. Comparing base (c46ed64) to head (2c34ede).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2233      +/-   ##
==========================================
+ Coverage   88.04%   88.05%   +0.01%     
==========================================
  Files         126      126              
  Lines        9963     9958       -5     
==========================================
- Hits         8772     8769       -3     
+ Misses        925      924       -1     
+ Partials      266      265       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

🟢 Approval recommended

The functional changes are low-risk and well-covered by new unit/e2e tests, with only a minor Help-text wording nit called out in review comments.

Pull request overview

This PR optimizes the resolver metrics hot path by eliminating per-query label-map allocations and expands/clarifies the per-client outcome metric (blocky_client_response_total) with supporting unit/e2e coverage and documentation updates.

Changes:

• Switches counter increments in MetricsResolver.Resolve to WithLabelValues(...) to avoid per-query prometheus.Labels map allocations.
• Adds/records blocky_client_response_total{client,response_type}, including a synthetic response_type="err" when no response is produced.
• Updates Prometheus/Grafana docs and strengthens unit + e2e tests to cover blocked/resolved/error paths and label behavior.

File summaries

File	Description
resolver/metrics_resolver.go	Avoids label-map allocations on the hot path; records new per-client outcome counter and defines response_type="err".
resolver/metrics_resolver_test.go	Adds unit specs for the new counter (resolved/blocked/error), multi-name client joining, and metrics-disabled behavior.
metrics/metrics_test.go	Extends registry completeness assertion to include blocky_client_response_total.
e2e/metrics_test.go	Validates the new counter appears in /metrics for both BLOCKED and RESOLVED outcomes.
docs/prometheus_grafana.md	Documents new metric semantics plus cardinality mitigation guidance and response_type value details.

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.