Skip to content

[sqlserver] Group perf counter rows by counter name before dispatch - #24876

Draft
eric-weaver wants to merge 1 commit into
masterfrom
eric.weaver/sqlserver-perf-counter-dispatch
Draft

[sqlserver] Group perf counter rows by counter name before dispatch#24876
eric-weaver wants to merge 1 commit into
masterfrom
eric.weaver/sqlserver-perf-counter-dispatch

Conversation

@eric-weaver

Copy link
Copy Markdown
Contributor

What does this PR do?

Restructures how rows from sys.dm_os_performance_counters are handed to metric objects, and fixes two related defects in the metric-list rebuild.

  • SqlSimpleMetric.fetch_all_values now strips the blank-padded name columns once and groups the rows into a dict keyed by counter name; fetch_metric looks up its own counter's rows instead of scanning the entire result set. This matches the contract SqlFractionMetric.fetch_all_values already uses in the same file. The ALL_INSTANCES, object_name, and first-match break semantics are unchanged, including the deliberate asymmetry where the ALL_INSTANCES branch does not apply the object_name filter.
  • get_sql_counter_type now caches the type of counters that need no base counter. The cache write previously sat inside the base-name branch, so the roughly 37 plain BULK_COUNT/LARGE_RAWCOUNT counters were never cached. A base-requiring counter whose base lookup fails is still left uncached so a transient failure is retried rather than pinned for the process lifetime.
  • _make_metric_list_to_collect builds instance_per_type_metrics into a local defaultdict(set) and assigns it at the end. It previously only ever added, so counter names from custom metrics that were removed stayed in the query's IN (...) list forever. Building locally and swapping in also means a rebuild that raises midway cannot leave this mapping inconsistent with instance_metrics.

No metrics are added, removed, or renamed, and no tags change.

Motivation

With autodiscovery, both the number of metric objects (40 + 8D) and the number of rows returned (one per database for the seven SQLServer:Databases counters) grow with the database count D, so dispatching every row past every metric object grew with D squared.

A microbenchmark over synthetic result sets padded to nchar(128), added here as sqlserver/tests/test_bench.py and run on the same machine before and after:

Databases Before After Speedup
50 24.0 ms 0.60 ms 40x
100 72.6 ms 1.92 ms 38x
250 442 ms 9.56 ms 46x
500 1739 ms 35.2 ms 49x

One caveat worth stating plainly: this is a large constant-factor win, not a change of complexity class. A per-database metric still walks its counter's rows until it finds its own database, so the residual cost is still quadratic in the database count, just with a roughly 40x smaller constant. Making it truly linear would mean keying the groups by instance name as well, which changes more of the matching logic and belongs in its own change.

The counter-type cache fix removes roughly 8D + 39 round trips per metric-list rebuild (about 840 at 100 databases), each of which created and closed its own cursor.

Note that dd.sqlserver.operation.time with operation:simple_metrics will not show the dispatch win, and may tick slightly up: tracked_query wraps only fetch_all_values, and the grouping work moves into that region while the dispatch loop it replaces was never inside it. Judge this by total check run duration.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

Every metric object scanned the whole sys.dm_os_performance_counters result
set, and with autodiscovery both the metric count and the row count grow with
the database count, so the dispatch cost grew with the square of the number of
databases. Group the rows by counter name once per run instead.

Also cache the counter type for counters that need no base counter, which was
costing a round trip per counter per database on every metric list rebuild, and
rebuild instance_per_type_metrics from scratch so counter names for metrics that
are no longer collected stop being queried.

Co-authored-by: Cursor <cursoragent@cursor.com>
@dd-octo-sts

dd-octo-sts Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Validation Report

All 21 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and code coverage settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
qa-label Validate the pull request declares whether it needs QA for the next Agent release
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

@datadog-official

datadog-official Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Tests  Code Coverage

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 90.62% (+2.11%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 092a017 | Docs | Datadog PR Page | Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant