[receiver/sqlquery] Add Connection Health Check Metric for Database Connectivity Status

[josepcorrea]

Component(s)

receiver/sqlquery

Is your feature request related to a problem? Please describe.

The current behavior of the sqlqueryreceiver lacks explicit observability for critical connection failures. If the receiver fails to establish a connection to the configured SQL database (DB), the expected business metrics simply stop being emitted, and the only indication of the failure is an error message logged within the Collector's logs.

This presents a significant observability gap because:

Passive Detection: Relying on the absence of metrics or requiring users to scrape and parse the Collector's internal logs makes failure detection slow, difficult to automate, and highly reactive.

Lack of Alerting Primitive: There is no dedicated explicit metric for "connection status" that can be used to set up immediate, high-priority, and reliable alerts in a monitoring platform (e.g., Prometheus, Grafana). We are frustrated that a critical dependency failure (database connectivity) only surfaces as a log entry instead of a dedicated, alertable signal.

Describe the solution you'd like

We propose extending the sqlqueryreceiver to include an optional configuration for a dedicated Connection Health Check. This check should attempt to establish/verify the connection to the DB and report the outcome as a standardized metric.

This metric should be a simple Gauge type, for instance:

Metric Name: sqlquery.connection.status

Value:

1 if the connection attempt to the database was successful.

0 if the connection attempt failed.

Attributes: The metric must include identifying attributes such as driver, host, and database (or the configured receiver name) to allow users to scope the alert accurately.

This connection status metric should be emitted regularly (ideally aligned with the collection interval) to ensure real-time status updates, allowing users to configure immediate alerts whenever the value drops to 0.

Describe alternatives you've considered

Monitoring for Metric Absence: The current, deficient solution. It introduces a high mean time to detect (MTTD) due to required "no-data" alerting rules.

Using a separate processor to convert logs to metrics: This approach is overly complex and resource-intensive (parsing Collector logs for specific connection errors) when the receiver itself has immediate knowledge of the failure. It increases pipeline fragility and complexity.

Configuring a dummy query (e.g., SELECT 1) as a regular metric: While functional, this only verifies query execution, not the underlying connection health check logic of the receiver itself. A dedicated health check metric is more explicit, robust, and clearer in intent.

Additional context

A dedicated connection health metric is a standard best practice across many telemetry components and would significantly enhance the operational visibility of the Collector pipeline. It cleanly separates failure modes:

Query Failure: Handled by existing query-specific metric logic (e.g., generating an error count).

Critical Connectivity Failure: Handled by the new dedicated sqlquery.connection.status metric.

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[github-actions]

Pinging code owners:

• receiver/sqlquery: @dmitryax @crobert-1

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[josepcorrea]

Hi @dmitryax , @crobert-1 :

As a follow-up suggestion, we could consider standardizing the implementation to match the behavior of the existing httpcheck receiver, which provides these metrics:

httpcheck_error{}
httpcheck_status{}

[axw]

Another possible option: have the receiver implement component status, and then monitor the component status more generally. That could either be done by polling the health check extension endpoint, or by enhancing the collector to produce metrics from component status (unless that exists?)

[josepcorrea]

Thanks @axw for the response.

Currently, I don’t see any internal telemetry metric that reflects the actual state or health of a receiver. The Collector only exposes operational metrics (accepted, refused, failed items, queue sizes, etc.), but nothing that explicitly indicates whether a receiver is connected or healthy.

The proposed approach of implementing component status at the receiver level is interesting, as it would provide a standardized health signal that could be used across all receivers. While this would increase the number of datapoints, it would offer a much clearer and more reliable way to monitor critical dependencies like database connections.

[crobert-1]

I think using the health check extension here would probably be the best option, as @axw pointed out. This issue can be used to track implementing the required functionality for the health check v2 extension. 👍

[github-actions]

Pinging code owners for extension/healthcheckv2: @mwear @evan-bradley. See Adding Labels via Comments if you do not have permissions to add labels yourself. For example, comment '/label priority:p2 -needs-triage' to set the priority and remove the needs-triage label.