|
| 1 | +--- |
| 2 | +title: Metrics SQL |
| 3 | +description: Query metrics views using SQL syntax |
| 4 | +sidebar_label: Metrics SQL |
| 5 | +--- |
| 6 | + |
| 7 | +You can write a SQL query referring to metrics definitions and dimensions defined in a metrics view. |
| 8 | +It should have the following structure: |
| 9 | + |
| 10 | +```yaml |
| 11 | +type: api |
| 12 | +metrics_sql: SELECT publisher, domain, total_records FROM ad_bids_metrics |
| 13 | +``` |
| 14 | +
|
| 15 | +## Querying Fundamentals |
| 16 | +
|
| 17 | +Metrics SQL transforms queries that reference `dimensions` and `measures` within a `metrics view` into their corresponding database columns or expressions. This transformation is based on the mappings defined in a metrics view YAML configuration, enabling reuse of dimension or measure definitions. Additionally, any security policies defined in the metrics view are also inherited. |
| 18 | + |
| 19 | +## Example: Crafting a Metrics SQL Query |
| 20 | + |
| 21 | +Consider a metrics view configured as follows: |
| 22 | +```yaml |
| 23 | +#metrics/ad_bids_metrics.yaml |
| 24 | +type: metrics_view |
| 25 | +title: Ad Bids |
| 26 | +model: ad_bids |
| 27 | +timeseries: timestamp |
| 28 | +dimensions: |
| 29 | + - name: publisher |
| 30 | + expression: toUpper(publisher) |
| 31 | + - name: domain |
| 32 | + column: domain |
| 33 | +measures: |
| 34 | + - name: total_records |
| 35 | + display_name: Total records |
| 36 | + expression: COUNT(*) |
| 37 | +``` |
| 38 | + |
| 39 | +To query this view, a user might write a Metrics SQL query like: |
| 40 | +```sql |
| 41 | +SELECT publisher, domain, total_records FROM ad_bids_metrics |
| 42 | +``` |
| 43 | + |
| 44 | +This Metrics SQL is internally translated to a standard SQL query as follows: |
| 45 | +```sql |
| 46 | +SELECT toUpper(publisher) AS publisher, domain AS domain, COUNT(*) AS total_records FROM ad_bids_metrics GROUP BY publisher, domain |
| 47 | +``` |
| 48 | + |
| 49 | +## Security and Compliance |
| 50 | + |
| 51 | +Queries executed via Metrics SQL are subject to the security policies and access controls defined in the metrics view YAML configuration, ensuring data security and compliance. |
| 52 | + |
| 53 | +## Limitations |
| 54 | + |
| 55 | +Metrics SQL is specifically designed for querying metrics views and may not support all features found in standard SQL. Its primary focus is on providing an efficient and easy way to extract data within the constraints of metrics view configurations. |
| 56 | + |
| 57 | +## Supported SQL Features |
| 58 | + |
| 59 | +- **SELECT** statements with plain `dimension` and `measure` references. |
| 60 | +- A single **FROM** clause referencing a `metrics view`. |
| 61 | +- **WHERE** clause that can reference selected `dimensions` only. |
| 62 | +- Operators in **WHERE** and **HAVING** clauses include `=`, `!=`, `>`, `>=`, `<`, `<=`, IN, LIKE, AND, OR, and parentheses for structuring the expression. |
| 63 | +- **HAVING** clause for filtering on aggregated results, referencing selected dimension and measure names. Supports the same expression capabilities as the WHERE clause. |
| 64 | +- **ORDER BY** clause for sorting the results. |
| 65 | +- **LIMIT** and **OFFSET** clauses for controlling the result set size and pagination. |
| 66 | + |
| 67 | +:::warning |
| 68 | + The Metrics SQL feature is currently evolving. We are dedicated to enhancing the syntax by introducing additional SQL features, while striving to maintain support for existing syntax. However, please be advised that backward compatibility cannot be guaranteed at all times. Additionally, users should be aware that there may be untested edge cases in the current implementation. We appreciate your understanding as we work to refine and improve this feature. |
| 69 | +::: |
| 70 | + |
| 71 | +## SQL Templating |
| 72 | + |
| 73 | +You can use templating to make your Metrics SQL query dynamic. We support: |
| 74 | + - Dynamic arguments that can be passed in as query parameters during the API call using `{{ .args.<param-name> }}` |
| 75 | + - User attributes like email, domain, and admin if available using `{{ .user.<attr> }}` (see integration docs [here](/integrate/custom-api) for when user attributes are available) |
| 76 | + - Conditional statements |
| 77 | + - Optional parameters paired with conditional statements. |
| 78 | + |
| 79 | +See integration docs [here](/integrate/custom-api) to learn how these are passed in when calling the API. |
| 80 | + |
| 81 | +### Conditional statements |
| 82 | + |
| 83 | +Assume an API endpoint defined as `my-api.yaml`: |
| 84 | +```yaml |
| 85 | +type: api |
| 86 | +metrics_sql: | |
| 87 | + SELECT publisher, total_records |
| 88 | + {{ if ( .user.admin ) }} ,domain {{ end }} |
| 89 | + FROM ad_bids_metrics WHERE timestamp::DATE = '{{ .args.date }}' |
| 90 | + {{ if ( .user.admin ) }} GROUP BY publisher, domain {{ else }} GROUP BY publisher {{ end }} |
| 91 | +``` |
| 92 | +If the user is an admin, the API will return the count of records by `publisher` and `domain` for the given date. If the user is not an admin, the API will return the total count of records by `publisher` for the given date. |
| 93 | + |
| 94 | +### Optional parameters |
| 95 | + |
| 96 | +Rill utilizes standard Go templating together with [Sprig](http://masterminds.github.io/sprig/), which adds a number of useful utility functions. |
| 97 | +One of those functions is `hasKey`, which in the example below enables optional parameters to be passed to the Custom API endpoint. This allows you to build API endpoints that can handle a wider range of parameters and logic, reducing the need to duplicate API endpoints. |
| 98 | + |
| 99 | +Assume an API endpoint defined as `my-api.yaml`: |
| 100 | +```yaml |
| 101 | +type: api |
| 102 | +metrics_sql: | |
| 103 | + SELECT |
| 104 | + publisher, |
| 105 | + total_records |
| 106 | + FROM ad_bids_metrics |
| 107 | + {{ if hasKey .args "publisher" }} WHERE publisher = '{{ .args.publisher }}' {{ end }} |
| 108 | + GROUP BY publisher |
| 109 | +``` |
| 110 | + |
| 111 | +HTTP GET `.../runtime/api/my-api` would return `total_records` for all `publisher`s. |
| 112 | +HTTP GET `.../runtime/api/my-api?publisher=Google` would return `total_records` for `Google`. |
| 113 | + |
| 114 | +## Add an OpenAPI spec |
| 115 | + |
| 116 | +You can optionally provide OpenAPI annotations for the request and response schema in your custom API definition. These will automatically be incorporated in the OpenAPI spec for your project (see [Custom API Integration](/integrate/custom-api) for details). |
| 117 | + |
| 118 | +Example custom API with request and response schema: |
| 119 | + |
| 120 | +```yaml |
| 121 | +type: api |
| 122 | +
|
| 123 | +metrics_sql: > |
| 124 | + SELECT publisher, total_records |
| 125 | + FROM ad_bids_metrics |
| 126 | + WHERE domain = '{{ .args.domain }}' |
| 127 | + {{ if hasKey .args "limit" }} LIMIT {{ .args.limit }} {{ end }} |
| 128 | + {{ if hasKey .args "offset" }} OFFSET {{ .args.offset }} {{ end }} |
| 129 | +
|
| 130 | +openapi: |
| 131 | + request_schema: |
| 132 | + type: object |
| 133 | + required: |
| 134 | + - domain |
| 135 | + properties: |
| 136 | + domain: |
| 137 | + type: string |
| 138 | + description: Domain to filter sales by |
| 139 | + limit: |
| 140 | + type: integer |
| 141 | + description: Optional limit for pagination |
| 142 | + offset: |
| 143 | + type: integer |
| 144 | + description: Optional offset for pagination |
| 145 | + |
| 146 | + response_schema: |
| 147 | + type: object |
| 148 | + properties: |
| 149 | + publisher: |
| 150 | + type: string |
| 151 | + description: Publisher name |
| 152 | + total_records: |
| 153 | + type: number |
| 154 | + description: Total records for the publisher |
| 155 | +``` |
| 156 | + |
| 157 | +## How to use Metrics SQL APIs |
| 158 | + |
| 159 | +Refer to the integration docs [here](/integrate/custom-api) to learn how to use Metrics SQL APIs in your application. |
| 160 | + |
0 commit comments