Skip to content

Conversation

@aalhour
Copy link

@aalhour aalhour commented Dec 10, 2025

PR Description

Add prometheus.exporter.databricks component for collecting metrics from Databricks System Tables including billing, jobs, pipelines, and SQL warehouse performance data.

Features:

  • OAuth2 M2M authentication via Service Principal
  • Configurable lookback windows for each metric domain
  • Optional high-cardinality task retry metrics
  • 19 metrics covering 3 personas: Finance, SRE/Platform, Analytics/BI

Arguments:

  • server_hostname, warehouse_http_path (required)
  • client_id, client_secret (required, secret)
  • query_timeout (default: 5m)
  • billing_lookback (default: 24h)
  • jobs_lookback, pipelines_lookback (default: 2h)
  • queries_lookback (default: 1h)
  • sla_threshold_seconds (default: 3600)
  • collect_task_retries (default: false)

Depends on: grafana/databricks-prometheus-exporter#14

Which issue(s) this PR fixes

Fixes: https://github.com/grafana/cloud-onboarding/issues/10029
Part of project: https://github.com/grafana/cloud-onboarding/issues/9716

Notes to the Reviewer

PR Checklist

  • CHANGELOG.md updated
  • Documentation added
  • Tests updated
  • Config converters updated

@CLAassistant
Copy link

CLAassistant commented Dec 10, 2025

CLA assistant check
All committers have signed the CLA.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 10, 2025

💻 Deploy preview available (WIP - feat(prometheus.exporter.databricks): add Databricks exporter component):

@aalhour aalhour self-assigned this Dec 10, 2025
Add prometheus.exporter.databricks component for collecting metrics from
Databricks System Tables including billing, jobs, pipelines, and SQL
warehouse performance data.

Features:
- OAuth2 M2M authentication via Service Principal
- Configurable lookback windows for each metric domain
- Optional high-cardinality task retry metrics
- 19 metrics covering 3 personas: Finance, SRE/Platform, Analytics/BI

Arguments:
- server_hostname, warehouse_http_path (required)
- client_id, client_secret (required, secret)
- query_timeout (default: 5m)
- billing_lookback (default: 24h)
- jobs_lookback, pipelines_lookback (default: 2h)
- queries_lookback (default: 1h)
- sla_threshold_seconds (default: 3600)
- collect_task_retries (default: false)

Depends on: github.com/grafana/databricks-prometheus-exporter
@aalhour aalhour force-pushed the bundle-databricks-exporter branch from b17d1d1 to ae221d2 Compare December 10, 2025 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants