Skip to content

feat(database): add mount point telemetry labels to database metrics - #32078

Open
vinanrra wants to merge 3 commits into
hashicorp:mainfrom
vinanrra:database-telemetry-mount-labels
Open

feat(database): add mount point telemetry labels to database metrics#32078
vinanrra wants to merge 3 commits into
hashicorp:mainfrom
vinanrra:database-telemetry-mount-labels

Conversation

@vinanrra

Copy link
Copy Markdown

Description

AI usage disclosure: The code, tests, and this pull request description were generated with the assistance of an AI tool. All changes have been reviewed by me before submission, and the affected test suites build and pass.

Part of #32077.

What does this PR do?

Database secrets engine metrics are currently keyed only by the plugin type string (for example, PostgreSQL emits database.pgx.*). There is no way to break them down by the mount, the namespace, or the configured connection, so every connection of a given type across every mount and namespace collapses into a single time series. This makes it impossible to answer questions like "which mount is driving credential churn?" or "is connection orders-primary slower than reporting-replica?".

This PR adds opt-in namespace, mount_point, and connection_name labels to the database plugin metrics, gated behind a new telemetry configuration option. Labels (rather than metric-name segments) are used deliberately, so existing dashboards and queries keep working unchanged — the same approach and rationale as the existing add_mount_point_rollback_metrics option.

Summary of changes

  • Add namespace, mount_point, and connection_name labels to database plugin metrics when the operator opts in via the new add_mount_point_database_metrics telemetry configuration.
  • Introduce a metricsLabelsForConnection method on the backend to build telemetry labels from the mount context (returns no labels unless opted in).
  • Thread labels through newDatabaseWrapperWithMetricsLabels (the existing newDatabaseWrapper signature is preserved as a thin wrapper for compatibility).
  • Extend the databaseBackend struct with mountNamespace, mountPoint, and includeMountPointInMetrics fields, populated from logical.BackendConfig.
  • Update the pluginInstances gauge collection to conditionally append namespace and mount_point labels, guarding against empty values.
  • Update the plugin metrics middleware and plugin factory to propagate the labels (new additive PluginFactoryConfig + PluginFactoryWithConfig; PluginFactory/PluginFactoryVersion signatures unchanged).
  • Add telemetry configuration plumbing (configutil.Telemetrymetricsutil.TelemetryConstConfigCorelogical.BackendConfig) plus a config-parse test and fixture.
  • Add test coverage in telemetry_labels_test.go that validates label emission against a real database backend and plugin process.

Operators can now correlate database plugin metrics across different mounts and namespaces for better observability.

Configuration

telemetry {
  add_mount_point_database_metrics = true
}

Behavior

Metric names are unchanged. Labels are added only when the option is enabled:

# default (option absent or false) — identical to today
database.pgx.Initialize

# with add_mount_point_database_metrics = true
database.pgx.Initialize;namespace=root;mount_point=db-prod/;connection_name=orders-primary

Affected series: the middleware operations (PluginVersion, Initialize, NewUser, UpdateUser, DeleteUser, Close) in both their database.<op> and database.<type>.<op> forms including the .error counters, plus the vault.secrets.database.backend.pluginInstances.count gauge.

Backward compatibility

  • Off by default. When disabled, emission is byte-for-byte identical to previous behavior (IncrCounterWithLabels(key, val, nil) is defined as IncrCounter(key, val) in go-metrics).
  • Public SDK function signatures (PluginFactory, PluginFactoryVersion) are unchanged; new inputs are added via an additive PluginFactoryConfig.
  • New logical.BackendConfig fields (MountPath, MountNamespace, IncludeMountPointInMetrics) are additive and optional.

Testing

  • Unit tests for the middleware verifying label presence, ordering (namespace, mount_point, connection_name), and that metric names are unchanged.
  • A full-stack test that mounts the database secrets engine, registers a real postgresql-database-plugin process, creates a connection, and asserts the labels appear only when opted in and are absent by default.
  • A telemetry config-parse test with fixture, an updated sanitized-config test, and a core mount-wiring test confirming the flag reaches BackendConfig.
  • gofmt clean; both the root and sdk modules build; affected suites pass.

Known limitations / follow-ups

  • The legacy v4 plugin path intentionally remains unlabeled (deprecated interface).
  • Enterprise-only connection-creation files are not in this (CE) tree; they compile as-is but need the same one-line change to emit labels on Enterprise builds.
  • The namespace label uses the telemetry-normalized value (root for the root namespace), consistent with other Vault per-mount metrics.

Related

TODO only if you're a HashiCorp employee

Not applicable — external/community contribution. Backport labeling and any Jira linkage are deferred to maintainers.

  • Backport Labels: If this fix needs to be backported, use the appropriate backport/ label that matches the desired release branch.

    • LTS: If this fixes a critical security vulnerability or severity 1 bug, it will also need to be backported to the current LTS versions of Vault. To ensure this, use all available enterprise labels.
  • Jira: If this change has an associated Jira, it's referenced either in the PR description, commit message, or branch name.

  • RFC: If this change has an associated RFC, please link it in the description.

PCI review checklist

  • I have documented a clear reason for, and description of, the change I am making.

  • If applicable, I've documented a plan to revert these changes if they require more than reverting the pull request.

    Not applicable — the change is stateless (no storage, schema, or data migrations). Reverting the pull request fully reverts the behavior; with the option disabled, no metric output changes at all.

  • If applicable, I've documented the impact of any changes to security controls.

    This change adds optional telemetry labels; it does not alter authentication, authorization, or any access-control path. When an operator explicitly enables add_mount_point_database_metrics, the mount path, namespace, and configured connection name become visible as metric labels wherever telemetry is already exposed (for example the sys/metrics endpoint or a configured sink). This is opt-in and off by default, metric names are unchanged, and time-series cardinality remains under operator control since it scales with the number of configured connections.

@vinanrra
vinanrra requested review from a team as code owners August 10, 2026 14:34
@vinanrra
vinanrra requested a review from Abdul5S August 10, 2026 14:34
@vercel

vercel Bot commented Aug 10, 2026

Copy link
Copy Markdown

Deployment failed for project vault-ui with the following error:

The `vercel.json` schema validation failed with the following message: should NOT have additional property `public`

Learn More: https://vercel.com/docs/concepts/projects/project-configuration

@hashicorp-cla-app

Copy link
Copy Markdown

CLA assistant check

Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement

Learn more about why HashiCorp requires a CLA and what the CLA includes


vfernandezg seems not to be a GitHub user.
You need a GitHub account to be able to sign the CLA.
If you have already a GitHub account, please add the email address used for this commit to your account.

Have you signed the CLA already but the status is still pending? Recheck it.

1 similar comment
@hashicorp-cla-app

Copy link
Copy Markdown

CLA assistant check

Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement

Learn more about why HashiCorp requires a CLA and what the CLA includes


vfernandezg seems not to be a GitHub user.
You need a GitHub account to be able to sign the CLA.
If you have already a GitHub account, please add the email address used for this commit to your account.

Have you signed the CLA already but the status is still pending? Recheck it.

@vercel

vercel Bot commented Aug 10, 2026

Copy link
Copy Markdown

Deployment failed for project vault-ui with the following error:

The `vercel.json` schema validation failed with the following message: should NOT have additional property `public`

Learn More: https://vercel.com/docs/concepts/projects/project-configuration

@vercel

vercel Bot commented Aug 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
vault-ui Error Error Aug 10, 2026 3:25pm

Request Review

@vercel

vercel Bot commented Aug 10, 2026

Copy link
Copy Markdown

Deployment failed for project vault-ui with the following error:

The `vercel.json` schema validation failed with the following message: should NOT have additional property `public`

Learn More: https://vercel.com/docs/concepts/projects/project-configuration

vfernandezg added 3 commits August 10, 2026 17:09
- Add mount namespace, mount point, and connection name labels to database plugin metrics when operator opts in via add_mount_point_database_metrics telemetry configuration
- Introduce metricsLabelsForConnection method to backend to generate telemetry labels based on mount context
- Update newDatabaseWrapper calls to pass metrics labels through newDatabaseWrapperWithMetricsLabels
- Extend databaseBackend struct with mountNamespace, mountPoint, and includeMountPointInMetrics fields
- Update gauge collection to conditionally append namespace and mount_point labels when configured
- Add comprehensive test coverage in telemetry_labels_test.go validating label emission with real database backend
- Update middleware and plugin factory to support labeled metrics propagation
- Add telemetry configuration helpers and test fixtures for database mount point metrics
- Operators can now correlate database plugin metrics across different mounts and namespaces for better observability
…t to changelog/32078.txt

- Align with standard changelog naming convention using issue number
…umentation

- Add validate() method to PluginFactoryConfig to check for required fields (Sys and Logger) before plugin initialization
- Prevent nil dereference panics by surfacing validation errors early to callers
- Refactor PluginFactoryWithConfig to use cfg directly instead of local variable assignments, improving consistency
- Update comments in backend.go to clarify mount point telemetry label behavior when opt-in is disabled
- Update telemetry.go documentation to reflect namespace labels in addition to mount_point and connection_name
- Add comprehensive test suite (plugin_factory_test.go) covering config validation scenarios
@vinanrra
vinanrra force-pushed the database-telemetry-mount-labels branch from c7f59b9 to 82b6eea Compare August 10, 2026 15:25
@vercel

vercel Bot commented Aug 10, 2026

Copy link
Copy Markdown

Deployment failed for project vault-ui with the following error:

The `vercel.json` schema validation failed with the following message: should NOT have additional property `public`

Learn More: https://vercel.com/docs/concepts/projects/project-configuration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core/metric core/telemetry enhancement secret/database size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant