You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/docs/integrations/libraries/openlineage.md
+18-7Lines changed: 18 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -53,16 +53,19 @@ event_log_storage:
53
53
postgres_url:
54
54
env: DAGSTER_PG_URL
55
55
namespace: my-company
56
-
# Optional:
57
-
# namespace_template: "{namespace}" # {tag:KEY} is not available in Mechanism A
56
+
# Optional — namespace_template overrides the static namespace field.
57
+
# Note: the {tag:KEY} token always resolves to an empty string in
58
+
# Mechanism A because EventLogStorage has no access to run tags at
59
+
# store_event time. Use Mechanism B if you need per-run tag resolution.
60
+
# namespace_template: "{namespace}"
58
61
# timeout: 2.0
59
62
```
60
63
61
64
Set `OPENLINEAGE_URL` (and optionally `OPENLINEAGE_API_KEY`) in the environment of any process that writes Dagster events — typically the run worker and the daemon.
62
65
63
66
## Mechanism B — sensor
64
67
65
-
Add `openlineage_sensor(include_asset_events=True)` to your `Definitions`:
68
+
Add `openlineage_sensor(include_asset_events=True)` to your `Definitions`. The sensor runs in the Dagster daemon and has full access to run tags, making it the right choice for `{tag:KEY}` namespace resolution and Dagster+ deployments.
66
69
67
70
```python
68
71
from dagster import Definitions
@@ -92,15 +95,23 @@ Set these environment variables on the process that runs the Dagster daemon:
92
95
93
96
## Namespace templates
94
97
95
-
Use `namespace_template` to route assets to per-tenant namespaces:
98
+
Use `namespace_template` to route assets to per-tenant namespaces. The `{tag:KEY}` token resolves to the run tag named `KEY`, and is only available in **Mechanism B** (the sensor has access to run tags). In Mechanism A it always resolves to an empty string.
96
99
97
-
```python
100
+
```
98
101
# Template: "{namespace}/{tag:tenant}"
99
102
# (Mechanism B, OPENLINEAGE_NAMESPACE=dagster)
100
103
# Run tags {"tenant": "acme"} → resolved namespace "dagster/acme"
If you are upgrading from `dagster-openlineage` v0.1 to v0.2, note the following breaking changes:
102
110
103
-
The `{tag:KEY}` token is available in Mechanism B (the sensor has access to run tags). In Mechanism A the token always resolves to an empty string because `EventLogStorage` has no access to run tags at `store_event` time.
111
+
-**Dagster version:** The minimum supported Dagster version is now `1.11.6`.
112
+
-**Default namespace:** The default namespace is now flat (`dagster`). In v0.1, the integration attempted to use the repository name as the namespace. If you wish to preserve the old behavior, configure the `namespace` or `namespace_template` option accordingly.
113
+
-**Removed class:** The legacy `OpenLineageEventListener` has been removed.
114
+
-**Emission mechanics:** v0.1 emitted pipeline and step events automatically with no extra configuration. In v0.2, you must explicitly configure exactly one mechanism (A or B). Both now include full asset-centric support.
0 commit comments