Skip to content

Commit b66ce5c

Browse files
feat(snowflake_workspace): parameterized tasks (WITH CONFIG) + multi-instance via assets_by_name
Same three-change pattern as v0.6.5 (procs), applied to tasks: 1. _make_task_asset() gained a config_v parameter. When non-empty, the asset emits: EXECUTE TASK <db>.<schema>.<task> WITH CONFIG => '<json>' where <json> is json.dumps(config_v) with single quotes escaped per Snowflake's literal rules. Inside the task body, SYSTEM$GET_TASK_GRAPH_CONFIG() exposes the JSON for SQL / Snowpark logic. When config_v is empty, the call is the plain EXECUTE TASK shape (backward-compatible with v0.6.5 and earlier). 2. Factory defined once before the per-task loop so both single-instance and multi-instance dispatch can call it (mirrors the proc fix). 3. Task-discovery loop now reads ``assets_by_name.<task>`` directly: ── Single-instance (default): one task → one Dagster asset, with an optional ``config:`` override. Backward-compatible with all existing configs (no override → no config → identical EXECUTE TASK <name> shape). ── Multi-instance: one task → N Dagster assets via an ``instances:`` list. Each instance can have its own ``config``, ``asset_name`` (or ``key``), ``description``, ``group_name``, ``deps``, ``kinds``, ``tags``, ``owners``. Lets customers wire per-tenant / per-region task invocations as N distinct catalog assets — the parent task definition stays singular in Snowflake while Dagster orchestrates N parameterized executions. The observation sensor's task_metadata bucket is populated for each instance (keyed by the instance's asset_name) so completion events land on the matching catalog node. YAML example for multi-instance: assets_by_name: REFRESH_REGION: instances: - asset_name: refresh_us config: {region: "US", batch_size: 1000} kinds: [snowflake, daily] - asset_name: refresh_eu config: {region: "EU", batch_size: 500} deps: [refresh_us] Both single + multi paths flow through the same _make_task_asset() factory and get the v0.6.4 numeric-perf-metadata enrichment for free. _apply_asset_overrides doesn't need touching — config: and instances: naturally fall through its allow-list (the override is read directly from self.assets_by_name above). L1 validation harness scorecard unchanged: PASS: 562 WARN: 0 FAIL: 41 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 0dde8ff commit b66ce5c

3 files changed

Lines changed: 195 additions & 110 deletions

File tree

dagster_community_components/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
import importlib.util
2222
from pathlib import Path
2323

24-
__version__ = "0.6.5"
24+
__version__ = "0.6.6"
2525

2626
_PACKAGE_ROOT = Path(__file__).resolve().parent
2727

0 commit comments

Comments
 (0)