Skip to content

Commit b1c4d3f

Browse files
Koenraad Verheydenknylander-grafanaelectron0zero
authored
Add documentation for user-configurable overrides module (#3374)
* Add documentation for user-configurable overrides module * reviewdog fixes * Tweaks/clean up * Update docs/sources/tempo/operations/user-configurable-overrides.md * Apply suggestions from code review Co-authored-by: Suraj Nath <[email protected]> * Update docs/sources/tempo/operations/user-configurable-overrides.md Co-authored-by: Kim Nylander <[email protected]> * Clarify returned -> take priority * Actually write out section conflicting overrides check * Update docs/sources/tempo/operations/user-configurable-overrides.md Co-authored-by: Kim Nylander <[email protected]> * Apply suggestions from code review --------- Co-authored-by: Kim Nylander <[email protected]> Co-authored-by: Suraj Nath <[email protected]>
1 parent 8514fc0 commit b1c4d3f

File tree

4 files changed

+231
-3
lines changed

4 files changed

+231
-3
lines changed

docs/sources/tempo/api_docs/_index.md

+4
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ For externally support GRPC API [see below](#tempo-grpc-api)
3232
| [Search tag values](#search-tag-values) | Query-frontend | HTTP | `GET /api/search/tag/<tag>/values` |
3333
| [Search tag values V2](#search-tag-values-v2) | Query-frontend | HTTP | `GET /api/v2/search/tag/<tag>/values` |
3434
| [Query Echo Endpoint](#query-echo-endpoint) | Query-frontend | HTTP | `GET /api/echo` |
35+
| [Overrides API](#overrides-api) | Query-frontend | HTTP | `GET,POST,PATCH,DELETE /api/overrides` |
3536
| Memberlist | Distributor, Ingester, Querier, Compactor | HTTP | `GET /memberlist` |
3637
| [Flush](#flush) | Ingester | HTTP | `GET,POST /flush` |
3738
| [Shutdown](#shutdown) | Ingester | HTTP | `GET,POST /shutdown` |
@@ -467,6 +468,9 @@ Returns status code 200 and body `echo` when the query frontend is up and ready
467468
Meant to be used in a Query Visualization UI like Grafana to test that the Tempo data source is working.
468469
{{% /admonition %}}
469470
471+
### Overrides API
472+
473+
For more information about user-configurable overrides API, refer to the [user-configurable overrides]{{< relref "../operations/user-configurable-overrides#api" >}} documentation.
470474
471475
### Flush
472476

docs/sources/tempo/configuration/_index.md

+48-3
Original file line numberDiff line numberDiff line change
@@ -1378,13 +1378,52 @@ overrides:
13781378
scope: <string> # scope of the attribute. options: resource, span
13791379
]
13801380

1381-
# Tenant-specific overrides settings configuration file. The empty string (default
1382-
# value) disables using an overrides file.
1383-
[per_tenant_override_config: <string> | default = ""]
1381+
# Tenant-specific overrides settings configuration file. The empty string (default
1382+
# value) disables using an overrides file.
1383+
[per_tenant_override_config: <string> | default = ""]
1384+
1385+
# How frequent tenant-specific overrides are read from the configuration file.
1386+
[per_tenant_override_period: <druation> | default = 10s]
1387+
1388+
# User-configurable overrides configuration
1389+
user_configurable_overrides:
1390+
1391+
# Enable the user-configurable overrides module
1392+
[enabled: <bool> | default = false]
1393+
1394+
# How often to poll the backend for new user-configurable overrides
1395+
[poll_interval: <duration> | default = 60s]
1396+
1397+
client:
1398+
# The storage backend to use
1399+
# Should be one of "gcs", "s3", "azure" or "local"
1400+
[backend: <string>]
1401+
1402+
# Backend-specific configuration, support the same configuration options as the
1403+
# trace backend configuration
1404+
local:
1405+
gcs:
1406+
s3:
1407+
azure:
1408+
1409+
# Check whether the backend supports versioning at startup. If enabled Tempo will not start if
1410+
# the backend doesn't support versioning.
1411+
[confirm_versioning: <bool> | default = true]
1412+
1413+
api:
1414+
# When enabled, Tempo will refuse request that modify overrides that are already set in the
1415+
# runtime overrides. For more details, see user-configurable overrides docs.
1416+
[check_for_conflicting_runtime_overrides: <bool> | default = false]
13841417
```
13851418
13861419
#### Tenant-specific overrides
13871420
1421+
There are two types of tenant-specific overrides:
1422+
- runtime overrides
1423+
- user-configurable overrides
1424+
1425+
##### Runtime overrides
1426+
13881427
You can set tenant-specific overrides settings in a separate file and point `per_tenant_override_config` to it.
13891428
This overrides file is dynamically loaded.
13901429
It can be changed at runtime and reloaded by Tempo without restarting the application.
@@ -1419,6 +1458,12 @@ overrides:
14191458
[max_bytes_per_trace: <int>]
14201459
```
14211460

1461+
##### User-configurable overrides
1462+
1463+
These tenant-specific overrides are stored in an object store and can be modified using API requests.
1464+
User-configurable overrides have priority over runtime overrides.
1465+
See [user-configurable overrides]{{< relref "../operations/user-configurable-overrides" >}} for more details.
1466+
14221467
#### Override strategies
14231468

14241469
The trace limits specified by the various parameters are, by default, applied as per-distributor limits.
Loading
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
---
2+
title: User-configurable overrides
3+
menuTitle: Configure Tempo overrides through the user-configurable overrides API
4+
description: Configure Tempo overrides through the user-configurable overrides API
5+
weight: 90
6+
---
7+
8+
# User-configurable overrides
9+
10+
User-configurable overrides in Tempo let you change overrides for your tenant using an API.
11+
Instead of modifying a file or Kubernetes configmap, you (and other services relying on Tempo) can use this API to modify the overrides directly.
12+
13+
### Architecture
14+
15+
![user-configurable-overrides-architecture.png](./user-configurable-overrides-architecture.png)
16+
17+
User-configurable overrides are stored in an object store bucket managed by Tempo.
18+
19+
{{% admonition type="note" %}}
20+
We recommend using a different bucket for overrides and traces storage, but they can share a bucket if needed.
21+
When sharing a bucket, make sure any lifecycle rules are scoped correctly to not remove data of user-configurable overrides module.
22+
{{% /admonition %}}
23+
24+
Overrides of every tenant are stored at `/{tenant name}/overrides.json`:
25+
26+
```
27+
overrides/
28+
├── 1/
29+
│ └── overrides.json
30+
└── 2/
31+
└── overrides.json
32+
```
33+
34+
Tempo regularly polls this bucket and keeps a copy of the limits in-memory. When requesting the overrides for a tenant, the overrides module:
35+
36+
1. Checks this override is set in the user-configurable overrides, if so return that value
37+
2. Checks if this override is set in the runtime config (configmap), if so return that value
38+
3. Returns the default value
39+
40+
### Supported fields
41+
42+
User-configurable overrides are designed to be a subset of the runtime overrides. Refer to [Overrides]{{< relref "../configuration/_index#overrides" >}} for information about all overrides.
43+
When a field is set in both the user-configurable overrides and the runtime overrides, the value from the user-configurable overrides will take priority.
44+
45+
{{% admonition type="note" %}}
46+
`processors` is an exception: Tempo will merge values from both user-configurable overrides and runtime overrides into a single list.
47+
{{% /admonition %}}
48+
49+
```yaml
50+
[forwarders: <list of strings>]
51+
52+
metrics_generator:
53+
54+
[processors: <list of strings>]
55+
[collection_interval: <duration>]
56+
[disable_collection: <bool> | default = false]
57+
58+
processor:
59+
60+
service_graphs:
61+
[histogram_buckets: <list of float>]
62+
[dimensions: <list of string>]
63+
[peer_attributes: <list of string>]
64+
[enable_client_server_prefix: <bool>]
65+
66+
span_metrics:
67+
[histogram_buckets: <list of float>]
68+
[dimensions: <list of string>]
69+
[intrinsic_dimensions: <map string to bool>]
70+
[filter_policies: [
71+
[
72+
include/exclude:
73+
match_type: <string> # options: strict, regexp
74+
attributes:
75+
- key: <string>
76+
value: <any>
77+
]
78+
]
79+
[enable_target_info: <bool>]
80+
[target_info_excluded_dimensions: <list of string>]
81+
```
82+
83+
### API
84+
85+
All API requests are handled on the `/api/overrides` endpoint. The module supports `GET`, `POST`, `PATCH`, and `DELETE` requests.
86+
87+
This endpoint is tenant-specific. If Tempo is run in multitenant mode, all requests should have an appropriate `X-Scope-OrgID` header.
88+
89+
If the tenant is run in distributed mode, only the query-frontend will accept API requests.
90+
91+
#### Operations
92+
93+
```
94+
GET /api/overrides
95+
```
96+
97+
Returns the current overrides and it's version.
98+
99+
Query-parameters:
100+
- `scope`: whether to return overrides from the API only `api` or merge it with the runtime overrides `merged`. Defaults to `api`.
101+
102+
Example: `curl -X GET -v -H "X-Scope-OrgID: 3" http://localhost:3100/tempo/api/overrides\?scope=merged`
103+
104+
```
105+
POST /api/overrides
106+
```
107+
108+
Update the overrides with the given payload. Note this will overwrite any existing overrides.
109+
110+
Example: `curl -X POST -v -H "X-Scope-OrgID: 3" -H "If-Match: 1697726795401423" http://localhost:3100/api/overrides --data "{}"
111+
112+
```
113+
PATCH /api/overrides
114+
```
115+
116+
Update the existing overrides by patching it with the payload. It follows the JSON merge patch protocol ([RFC 7386](https://datatracker.ietf.org/doc/html/rfc7386)).
117+
118+
Example: `curl -X PUT -v -H "X-Scope-OrgID: 3" -H "If-Match: 1697726795401423" http://localhost:3100/api/overrides --data "{\"forwarders\":null}"
119+
120+
```
121+
DELETE /api/overrides
122+
```
123+
124+
Delete the existing overrides.
125+
126+
Example: `curl -X DELETE -H "X-Scope-OrgID: 3" -H "If-Match: 1697726795401423" http://localhost:3100/api/overrides`
127+
128+
#### Versioning
129+
130+
To handle concurrent read and write operations, overrides are stored with a version in the backend.
131+
Whenever the overrides are returned, the response will have an Etag header with the current version.
132+
133+
```
134+
$ curl -v http://localhost:3100/api/overrides
135+
...
136+
< HTTP/1.1 200 OK
137+
< Content-Type: application/json
138+
< Etag: 1697726795401423
139+
< Date: Wed, 07 Feb 2024 17:49:04 GMT
140+
< Content-Length: 118
141+
...
142+
```
143+
144+
Requests that modify or delete overrides need to pass the current version using the `If-Match` header:
145+
146+
```
147+
$ curl -X POST -H "If-Match: 1697726795401423" http://localhost:3100/api/overrides --data "..."
148+
```
149+
This example uses uses overrides in the `overrides.json` file with the location in `pwd`:
150+
151+
`curl -X POST -H "X-Scope-OrgID: 3" -H "If-Match: 1697726795401423" http://localhost:3100/api/overrides --data @overrides.json`
152+
153+
If the version does not match the version in the backend, the request is rejected with HTTP error 412.
154+
155+
#### Conflicting runtime overrides check
156+
157+
Overrides set through the user-configurable overrides take priority over runtime overrides.
158+
This can lead to misleading scenarios because a value set in the runtime overrides is not actively being used.
159+
160+
To warn users about preexisting runtime overrides, there is an optional check for conflicting runtime overrides.
161+
If enabled requests are rejected if:
162+
163+
1. There are no user-configurable overrides yet for this tenant.
164+
2. There are runtime overrides set that contain overrides present in the user-configurable overrides.
165+
166+
The check can be enabled in the configuration:
167+
168+
```yaml
169+
overrides:
170+
user_configurable_overrides:
171+
api:
172+
check_for_conflicting_runtime_overrides: true
173+
```
174+
175+
You can bypass this check by setting the query parameter `skip-conflicting-overrides-check=true`:
176+
177+
```
178+
$ curl -X POST -H "If-Match: 1697726795401423" http://localhost:3100/api/overrides?skip-conflicting-overrides-check=true --data "..."
179+
```

0 commit comments

Comments
 (0)