You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(cron_schedule): opt-in time-partitioned-job support (v0.8.2)
ScheduleDefinition errors on every tick against a partitioned asset job
with "no partition key provided". Add an opt-in path that switches to
build_schedule_from_partitioned_job when any time-based partition field
is set, so each tick auto-targets the most-recent finished partition.
- New fields (mirroring the synthetic_data_generator + sinks shape):
partition_type, partition_start, partition_values,
partition_dimensions, dynamic_partition_name.
- cron_expression becomes Optional in the partitioned path — Dagster
infers cadence from partitions_def. If supplied, validated against
the partitions_def cadence (raises ValueError on mismatch instead
of silently misfiring).
- partition_type='static' / 'dynamic' raise ValueError in the
partitioned path — build_schedule_from_partitioned_job is
time-only.
No change to the un-partitioned path: when neither partition_type nor
partition_dimensions is set, behavior is byte-for-byte identical.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: schedules/cron_schedule/README.md
+41-1Lines changed: 41 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,6 @@ Defines a `ScheduleDefinition` + an underlying asset job that materializes the c
11
11
| Field | Type | Description |
12
12
|---|---|---|
13
13
|`schedule_name`|`str`| Unique schedule name shown in the UI. |
14
-
|`cron_expression`|`str`| Cron expression (5 fields: m h dom mon dow). E.g. '0 9 * * *' = daily 09:00. |
15
14
|`asset_keys`|`List[str]`| Slash-separated asset keys to materialize on each tick. |
16
15
17
16
### Catalog metadata
@@ -20,6 +19,15 @@ Defines a `ScheduleDefinition` + an underlying asset job that materializes the c
20
19
|---|---|---|---|
21
20
|`tags`|`Dict[str, str]`| — | Tags applied to runs created by this schedule. |
22
21
22
+
### Partitions
23
+
24
+
| Field | Type | Default | Description |
25
+
|---|---|---|---|
26
+
|`partition_type`|`str`| — | Partition type: 'daily', 'weekly', 'monthly', 'hourly', 'static', 'multi', 'dynamic', or None for unpartitioned. Note: only time-based types ('daily'/'weekly'/'monthly'/'hourly') work with this schedule's partitioned-job path — 'static' / 'dynamic' raise ValueError. |
27
+
|`partition_start`|`str`| — | Partition start in ISO format (e.g. '2024-01-01' or '2024-01-01-00:00' for hourly). Required for time-based types. |
28
+
|`partition_values`|`str`| — | Comma-separated values for static or multi partitioning. Only used if you also set partition_type='multi' (and even then the schedule rejects pure static — see partition_type). |
29
+
|`partition_dimensions`|`List[Dict[str, Any]]`| — | Multi-axis partition spec: list of {name, type, start, values, dynamic_partition_name} dicts. Overrides flat fields when set. |
30
+
23
31
### Sensor configuration
24
32
25
33
| Field | Type | Default | Description |
@@ -31,7 +39,9 @@ Defines a `ScheduleDefinition` + an underlying asset job that materializes the c
31
39
32
40
| Field | Type | Default | Description |
33
41
|---|---|---|---|
42
+
|`cron_expression`|`str`| — | Cron expression (5 fields: m h dom mon dow). E.g. '0 9 * * *' = daily 09:00. REQUIRED when no partition fields are set; OPTIONAL in the partitioned-job path (cadence is inferred from the partitions_def). If supplied in the partitioned path, validated against the partitions_def cadence. |
34
43
|`execution_timezone`|`str`|`"UTC"`| IANA timezone for cron evaluation, e.g. 'America/Los_Angeles'. |
44
+
|`dynamic_partition_name`|`str`| — | Name for DynamicPartitionsDefinition (when partition_type='dynamic'). NOTE: the schedule rejects dynamic-only partitions. |
35
45
36
46
<!-- FIELDS:END -->
37
47
@@ -47,3 +57,33 @@ attributes:
47
57
tags:
48
58
team: data-platform
49
59
```
60
+
61
+
### Partitioned-job mode
62
+
63
+
When any time-based partition field (`partition_type`,
64
+
`partition_dimensions`) is set, the component switches from
65
+
`ScheduleDefinition`to `build_schedule_from_partitioned_job`. Each
66
+
tick auto-targets the most-recent finished partition — without this
67
+
switch, `ScheduleDefinition` against a partitioned asset job errors
68
+
with "no partition key provided" on every tick.
69
+
70
+
`cron_expression`is OPTIONAL in this mode (Dagster infers cadence
71
+
from the partitions_def). If you supply one, it's validated against
72
+
the partitions_def cadence — a daily cron paired with hourly
73
+
partitions raises `ValueError` at load time rather than failing
0 commit comments