Skip to content

Commit 6012c6f

Browse files
lukekimpeasee
andauthored
Improve the SQL reference (#986)
* Improve the SQL reference - Add proper index - Add Scalar Functions section * Tweaks * Remove extra reference * Fix v1.2.1 * Fix headings * Tweaks * fix: SQL reference sidebar link * fix: Remove leading slash * fix: Remove index.md from path * WIP * Link updates --------- Co-authored-by: peasee <98815791+peasee@users.noreply.github.com>
1 parent 82287e9 commit 6012c6f

19 files changed

Lines changed: 4404 additions & 2887 deletions

File tree

.github/workflows/build_and_publish.yml

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,12 +24,10 @@ jobs:
2424
pull-requests: write
2525
steps:
2626
- uses: actions/checkout@v4
27-
with:
28-
fetch-depth: 0
2927

3028
- uses: actions/setup-node@v4
3129
with:
32-
node-version: 18
30+
node-version: 24
3331

3432
- name: Install dependencies
3533
working-directory: website

website/docs/api/jdbc/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ Follow the instructions specific to your application for adding a custom JDBC dr
4545
- In the DBeaver application menu bar, open the "Database" menu and choose: "Driver Manager"
4646
- Click the "New" button and follow instructions to add JDBC `.jar` file.
4747

48-
[Full instruction](/docs/clients/DBeaver)
48+
[Full instruction](/docs/clients/dbeaver)
4949

5050
## Configure JDBC connection
5151

website/docs/clients/DBeaver/index.md

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -22,33 +22,26 @@ pagination_next: null
2222
![Driver manager new button](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/5783d944-daae-4735-99e9-976f974bc100/public 'Driver manager new button')
2323

2424
7. Add the JDBC jar file:
25-
2625
1. Click the "Libraries" tab
2726
1. Click the: "Add File" button
2827
1. Choose the "flight-sql-jdbc-driver-15.0.1.jar" jar file (the file downloaded in step 3 above) - and click "Open"
2928
![Select jar file](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/19900f7a-f00f-473d-780e-4a28c2ecd800/public 'Select jar file')
3029
1. Close the Driver editor window with the blue "OK" button on the lower-right
3130

3231
8. Enter the driver settings:
33-
3432
1. Click the "Settings" tab
3533
1. In the "Driver Name" field - enter: `Apache Arrow Flight SQL`
3634
1. In the "URL Template" field - enter: `jdbc:arrow-flight-sql://{host}:{port}?useEncryption=false&disableCertificateVerification=true`
37-
3835
- If [API key authentication](../../api/auth/index.md) is enabled, the URL template should be: `jdbc:arrow-flight-sql://{host}:{port}?useEncryption=false&disableCertificateVerification=true&user=&password=<enter-api-key-here>` - where `<enter-api-key-here>` is the API key value
39-
4036
1. In the "Driver Type" drop-down box - choose: "SQLite"
4137
1. Select "No authentication"
42-
4338
- This should be selected even if API key authentication is enabled in the runtime, as the API key is supplied via the URL template above.
44-
4539
1. The driver manager "Edit Driver" window should look like this:
4640
![Driver Manager completed](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/20348c42-117b-4763-80d2-6e615b23ae00/public 'Driver Manager completed')
4741
1. Click the blue "OK" button on the lower-right to save the driver
4842
1. Close the "Driver Manager" window by clicking the blue "Close" button on the lower-right.
4943

5044
9. Create a new Database Connection:
51-
5245
1. In the DBeaver application menu bar, open the "Database" menu and choose: "New Database Connection":
5346
![New Database Connection](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/acdf7251-4238-44ee-9639-0c557518da00/public 'New Database Connection')
5447
1. In the "Connect to a database" window - type: `Flight` in the search bar

website/docs/clients/Datadog/index.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
2-
title: "Datadog"
3-
sidebar_label: "Datadog"
2+
title: 'Datadog'
3+
sidebar_label: 'Datadog'
44
description: 'Monitoring Spice with Datadog'
55
pagination_prev: 'clients/index'
66
sidebar_position: 3
@@ -21,10 +21,10 @@ Configure the Datadog Agent to scrape the Spice metrics endpoint:
2121
init_config:
2222

2323
instances:
24-
- prometheus_url: SPICE-METRICS-ENDPOINT>/metrics # for example http://localhost:9090/metrics
25-
namespace: spice
26-
metrics:
27-
- '*'
24+
- prometheus_url: SPICE-METRICS-ENDPOINT>/metrics # for example http://localhost:9090/metrics
25+
namespace: spice
26+
metrics:
27+
- '*'
2828
```
2929
3030
1. [Restart the Agent](https://docs.datadoghq.com/agent/guide/agent-commands/#start-stop-and-restart-the-agent) to start collecting Spice metrics.
@@ -45,4 +45,4 @@ instances:
4545

4646
3. Dashbord is now configured to display Spice.ai OSS key performance metrics
4747

48-
<img width="800" src="/img/datadog/spice_datadog_dashboard.png"/>
48+
<img width="800" src="/img/datadog/spice_datadog_dashboard.png"/>
Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
---
2+
title: 'Datadog'
3+
sidebar_label: 'Datadog'
4+
description: 'Monitoring Spice with Datadog'
5+
pagination_prev: 'clients/index'
6+
sidebar_position: 3
7+
pagination_next: null
8+
---
9+
10+
Spice can be monitored with [Datadog](https://www.datadoghq.com/) using the [Spice Metrics Endpoint](/docs/features/observability/) and pre-built dashboards available in the [Spice repository](https://github.com/spiceai/spiceai/tree/trunk/monitoring).
11+
12+
## Datadog Agent Configuration
13+
14+
Prerequisite: [Datadog Agent version 6.5.0 or later is installed](https://docs.datadoghq.com/getting_started/agent/).
15+
16+
Configure the Datadog Agent to scrape the Spice metrics endpoint:
17+
18+
1. Edit the `openmetrics.d/conf.yaml` file in the `conf.d/` folder at the root of your [Agent’s configuration directory](https://docs.datadoghq.com/agent/guide/agent-configuration-files/#agent-configuration-directory):
19+
20+
```yaml
21+
init_config:
22+
23+
instances:
24+
- prometheus_url: SPICE-METRICS-ENDPOINT>/metrics # for example http://localhost:9090/metrics
25+
namespace: spice
26+
metrics:
27+
- '*'
28+
```
29+
30+
1. [Restart the Agent](https://docs.datadoghq.com/agent/guide/agent-commands/#start-stop-and-restart-the-agent) to start collecting Spice metrics.
31+
1. Refer to [Prometheus and OpenMetrics metrics collection from a host](https://docs.datadoghq.com/integrations/guide/prometheus-host-collection/) for all available configuration options and supported parameters.
32+
1. Open Datadog Metrics Explorer and type `spice` to confirm Spice telemetry information is successfully collected.
33+
34+
<img width="800" src="/img/datadog/spice_datadog_metrics_explorer.png"/>
35+
36+
## Import the Spice Datadog Dashboard
37+
38+
1. Create [New Datadog Dashboard](https://docs.datadoghq.com/dashboards/#get-started)
39+
40+
<img width="800" src="/img/datadog/spice_datadog_dashboard_new.png"/>
41+
42+
2. Click **Import dashboard JSON** and drag and drop [monitoring/datadog-dashboard.json](https://raw.githubusercontent.com/spiceai/spiceai/trunk/monitoring/datadog-dashboard.json) file
43+
44+
<img width="800" src="/img/datadog/spice_datadog_dashboard_import.png"/>
45+
46+
3. Dashbord is now configured to display Spice.ai OSS key performance metrics
47+
48+
<img width="800" src="/img/datadog/spice_datadog_dashboard.png"/>
Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
---
2+
title: 'DBeaver'
3+
sidebar_label: 'DBeaver'
4+
description: 'Configure DBeaver to query Spice via JDBC'
5+
sidebar_position: 2
6+
pagination_prev: 'clients/index'
7+
pagination_next: null
8+
---
9+
10+
1. Start the Spice runtime with a dataset loaded. Follow the [quickstart guide](/docs/getting-started) to get started.
11+
12+
2. Download [DBeaver Community Edition](https://dbeaver.io).
13+
14+
3. Download the [Apache Arrow Flight SQL JDBC driver](https://search.maven.org/search?q=a:flight-sql-jdbc-driver) - choose the "jar" option.
15+
16+
4. Launch DBeaver
17+
18+
5. In the DBeaver application menu bar, open the "Database" menu and choose: "Driver Manager":
19+
![Driver manager menu option](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/691d1f83-c1d0-4ad8-ec8d-d8f37ccc9d00/public 'Driver manager menu option')
20+
21+
6. Click the "New" button on the right:
22+
![Driver manager new button](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/5783d944-daae-4735-99e9-976f974bc100/public 'Driver manager new button')
23+
24+
7. Add the JDBC jar file:
25+
1. Click the "Libraries" tab
26+
1. Click the: "Add File" button
27+
1. Choose the "flight-sql-jdbc-driver-15.0.1.jar" jar file (the file downloaded in step 3 above) - and click "Open"
28+
![Select jar file](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/19900f7a-f00f-473d-780e-4a28c2ecd800/public 'Select jar file')
29+
1. Close the Driver editor window with the blue "OK" button on the lower-right
30+
31+
8. Enter the driver settings:
32+
1. Click the "Settings" tab
33+
1. In the "Driver Name" field - enter: `Apache Arrow Flight SQL`
34+
1. In the "URL Template" field - enter: `jdbc:arrow-flight-sql://{host}:{port}?useEncryption=false&disableCertificateVerification=true`
35+
- If [API key authentication](../../api/auth/index.md) is enabled, the URL template should be: `jdbc:arrow-flight-sql://{host}:{port}?useEncryption=false&disableCertificateVerification=true&user=&password=<enter-api-key-here>` - where `<enter-api-key-here>` is the API key value
36+
1. In the "Driver Type" drop-down box - choose: "SQLite"
37+
1. Select "No authentication"
38+
- This should be selected even if API key authentication is enabled in the runtime, as the API key is supplied via the URL template above.
39+
1. The driver manager "Edit Driver" window should look like this:
40+
![Driver Manager completed](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/20348c42-117b-4763-80d2-6e615b23ae00/public 'Driver Manager completed')
41+
1. Click the blue "OK" button on the lower-right to save the driver
42+
1. Close the "Driver Manager" window by clicking the blue "Close" button on the lower-right.
43+
44+
9. Create a new Database Connection:
45+
1. In the DBeaver application menu bar, open the "Database" menu and choose: "New Database Connection":
46+
![New Database Connection](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/acdf7251-4238-44ee-9639-0c557518da00/public 'New Database Connection')
47+
1. In the "Connect to a database" window - type: `Flight` in the search bar
48+
1. Choose the `Apache Arrow Flight SQL` driver - the window should look like this:
49+
![Connect to a database window](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/61cee5fe-dc75-4ac1-e558-eea3aff4c100/public 'Connect to a database window')
50+
1. Click the blue "Next >" button on the bottom of the window
51+
1. On the next screen, the JDBC URL should be filled out already - just supply the Host (`localhost`) and Port (`50051`) values for the Spice runtime. The window should look like this:
52+
![Connect to a database window 2](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/2a2b2fdc-00db-49d3-5359-059b12342b00/public 'Connect to a database window 2')
53+
1. Click the "Test Connection" button - the window should look like this:
54+
![Test Connection results](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/a3fc5f5f-a39f-47ce-7955-4b384ec1ae00/public 'Test Connection results')
55+
1. Click the blue "OK" button to close the Connection test window
56+
1. Click the "Connection details (name, type, ...)" button on the right
57+
1. In the "General" section, enter: `Spice Runtime` for the "Connection name". It should look like this:
58+
![Name the Database Connection](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/f6d04fe1-92a1-4082-d4ea-e9daacaca200/public)
59+
1. Click the blue "Finish" button to save the connection
60+
61+
10. Run a query:
62+
1. Right-click on the Database Connection on the left - choose: "SQL Editor", and then: "Open SQL Console" as shown here:
63+
![Open SQL Console](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/642a5885-9e3f-4dd7-ef43-72bfce27bb00/public 'Open SQL Console')
64+
1. In the Console window - run a query - something like: `SELECT * FROM taxi_trips;`
65+
1. Click the triangle button to execute the SQL statement - as shown below (or use keyboard shortcut: Ctrl+Enter):
66+
![Execute SQL](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/2134e47b-a066-47e9-1d48-06352675f400/public 'Execute SQL')
67+
1. See the query results as shown in this screenshot:
68+
![Query Results](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/0e9f3c0f-2e03-47f9-8d5e-65e078d7e900/public 'Query Results')
69+
1. DBeaver is now configured to query the Spice runtime using SQL! 🎉

0 commit comments

Comments
 (0)