feat: update Databricks quickstart to reflect changes in latest version of integration

data-sources/databricks/config.yml

@@ -1,12 +1,15 @@
 id: databricks
 displayName: Databricks Integration
 description: |
-  This integration collects Spark telemetry, Workflow telemetry, and Cost and Billing information from Databricks.
+  The Databricks Integration collects Apache Spark application metrics,
+  Databricks Lakeflow job run metrics, Databricks Lakeflow Spark Declarative
+  Pipeline update metrics, Databricks query metrics, Databricks cluster health
+  metrics and logs, and Databricks consumption and cost data.
 icon: logo.png
 install:
   primary:
     link:
-      url: https://github.com/newrelic-experimental/newrelic-databricks-integration
+      url: https://github.com/newrelic/newrelic-databricks-integration?tab=readme-ov-file#getting-started
 keywords:
   - nrlabs
   - nrlabs-data

quickstarts/databricks/config.yml

@@ -1,16 +1,38 @@
 id: 533cdd19-8232-42cb-b134-e7d17bfff581
 slug: databricks
-title: Databricks Spark Integration
+title: Databricks Integration
 description: |
-  Databricks is an orchestration platform for Apache Spark. Instantly monitor Databricks Spark clusters with our New Relic Spark integration.
-  This integration collects Spark telemetry, Workflow telemetry, and Cost and Billing information from Databricks.
+  ## Why monitor Databricks
 
-  The New Relic Databricks integration can collect telemetry from Spark running on Databricks. By default, the integration will automatically connect to and collect telemetry from the Spark deployments in all clusters created via the UI or API in the specified workspace.
+  In the world of big data, Databricks is a mission-critical platform. But
+  making sure your workloads are running efficiently, cost-effectively, and
+  reliably can be challenging.
+
+  The Databricks Integration from New Relic delivers total visibility for your
+  entire Databricks estate, allowing you to troubleshoot, optimize, and connect
+  performance directly to cost - all from a single, unified observability
+  platform.
+
+  ### Databricks quickstart highlights
+
+  The Databricks integration collects a comprehensive suite of telemetry data,
+  including the following:
+
+  - Apache Spark application metrics
+  - Databricks Lakeflow job run metrics
+  - Databricks Lakeflow Spark Declarative Pipeline update metrics
+  - Databricks query metrics
+  - Databricks cluster health metrics and logs
+  - Databricks consumption and cost data
+
+  With the pre-built dashboards in this quickstart, you can quickly visualize
+  and analyze your Databricks workloads to ensure optimal performance and cost
+  efficiency.
 summary: |
-  Monitor Databricks Spark clusters with the New Relic Databricks integration
+  Gain full visibility into your entire Databricks estate with the comprehensive
+  suite of telemetry data collected by the Databricks Integration
 icon: logo.png
 level: Community
-
 keywords:
   - nrlabs
   - nrlabs-data
@@ -23,11 +45,22 @@ keywords:
 authors:
   - New Relic Labs
 documentation:
-  - name: Databricks integration docs
+  - name: Getting Started
+    description: |
+      Follow our Getting Started documentation to quickly instrument your
+      Databricks environment and start visualizing your data in New Relic.
+    url: https://github.com/newrelic/newrelic-databricks-integration?tab=readme-ov-file#getting-started
+  - name: Learn More
     description: |
-      Collect Spark telemetry data with the New Relic Databricks integration
-    url: https://github.com/newrelic-experimental/newrelic-databricks-integration
+      Learn more about the Databricks Integration in our Usage Guide, including
+      configuration options, telemetry collected, and dashboards included.
+    url: https://github.com/newrelic/newrelic-databricks-integration?tab=readme-ov-file#usage-guide
 dataSourceIds:
   - databricks
 dashboards:
   - databricks-spark
+  - databricks-job-runs
+  - databricks-pipeline-updates
+  - databricks-query-metrics
+  - databricks-cluster-health
+  - databricks-consumption-cost