Skip to content

Commit 9d15a66

Browse files
OwenKephartDagster Devtools
authored andcommitted
Add asset check and partition status query commands to dg api (#22132)
Implement new query capabilities for asset checks and partition status, adding missing commands that `dagctl` supports. Introduces a new `dg api asset-check` command group with `list` and `get-executions` subcommands, plus `dg api asset get-partition-status` for partition materialization stats. **Objective #22131:** Objective: Add missing query commands to dg api ## Key Changes - New `asset-check` command group with `list` and `get-executions` subcommands for querying check metadata and execution history - New `asset get-partition-status` command to fetch partition materialization statistics (materialized, failed, in-progress, total) - Complete 4-layer implementation across schemas, GraphQL adapter, API, and CLI layers following existing patterns - Comprehensive test coverage with snapshot tests for formatting and dynamic command execution recording - GraphQL queries for `assetChecksOrError`, `assetCheckExecutions`, and asset partition stats with proper error handling <details> <summary>Files Changed</summary> ### Added (9 files) - `dagster-oss/.../dagster_dg_cli/api_layer/api/asset_check.py` - Asset check API with list and get_check_executions methods - `dagster-oss/.../dagster_dg_cli/api_layer/graphql_adapter/asset_check.py` - GraphQL queries and response processors for asset checks - `dagster-oss/.../dagster_dg_cli/api_layer/schemas/asset_check.py` - Pydantic models for checks and executions - `dagster-oss/.../dagster_dg_cli/cli/api/asset_check.py` - Click commands for asset-check group - `dagster_dg_cli_tests/cli_tests/api_tests/asset_check_tests/` - Complete test suite with scenarios, snapshots, and recordings ### Modified (6 files) - `asset.py` (schemas) - Added DgApiPartitionStats model - `asset.py` (graphql_adapter) - Added ASSET_PARTITION_STATUS_QUERY and get_asset_partition_status_via_graphql function - `asset.py` (api) - Added get_partition_status method to DgApiAssetApi - `asset.py` (cli) - Added get-partition-status command to asset group - `cli_group.py` - Registered asset-check command group - `formatters.py` - Added format_asset_checks, format_asset_check_executions, format_partition_status </details> ## User Experience **Before:** ``` $ dg api asset-check list --asset-key my/asset Error: No such command 'asset-check' ``` **After:** ``` $ dg api asset-check list --asset-key my/asset NAME BLOCKING DESCRIPTION freshness_check Yes Checks that the asset is materialized within the last 24 ... row_count_check No Validates minimum row count $ dg api asset get-partition-status my/asset Materialized 10 Failed 1 In Progress 0 Total 20 ``` Users can now query asset checks and partition materialization stats directly from the CLI, matching the capabilities previously only available in `dagctl`. <details> <summary>original-plan</summary> # Plan: Asset checks + partition status commands Part of Objective #22131, Steps 1.1–1.3 ## Context `dg api` is missing query commands that `dagctl` supports: listing asset checks, getting check execution history, and getting partition materialization stats. This PR adds a new `dg api asset-check` top-level command group and a `dg api asset get-partition-status` subcommand. ## Changes ### 1. New `asset-check` command group (Steps 1.1 + 1.2) Create 4 new files following the existing 4-layer pattern (schedule/sensor are the closest models): **Schema** — `dagster_dg_cli/api_layer/schemas/asset_check.py` - `DgApiAssetCheck` — name, asset_key, description, blocking, job_names, can_execute_individually - `DgApiAssetCheckList` — items list - `DgApiAssetCheckExecutionStatus` enum — IN_PROGRESS, SUCCEEDED, FAILED, SKIPPED, EXECUTION_FAILED - `DgApiAssetCheckExecution` — id, run_id, status, timestamp, partition, step_key, check_name, asset_key - `DgApiAssetCheckExecutionList` — items list with cursor/has_more **GraphQL adapter** — `dagster_dg_cli/api_layer/graphql_adapter/asset_check.py` - `ASSET_CHECKS_QUERY` — query `assetChecksOrError(assetKey)` returning check name, description, blocking, jobNames, canExecuteIndividually - `ASSET_CHECK_EXECUTIONS_QUERY` — query `assetCheckExecutions(assetKey, checkName, limit, cursor)` returning execution details - Process functions for each query **API class** — `dagster_dg_cli/api_layer/api/asset_check.py` - `DgApiAssetCheckApi(client: IGraphQLClient)` frozen dataclass - `list_asset_checks(asset_key: str)` method - `get_check_executions(asset_key: str, check_name: str, limit: int, cursor: str | None)` method **CLI** — `dagster_dg_cli/cli/api/asset_check.py` - `dg api asset-check list --asset-key ASSET_KEY [--json]` - `dg api asset-check get-executions --asset-key ASSET_KEY --check-name CHECK_NAME [--limit N] [--cursor C] [--json]` - `asset_check_group` click group **Formatters** — add to `dagster_dg_cli/cli/api/formatters.py` - `format_asset_checks(checks, as_json)` — table: NAME | BLOCKING | DESCRIPTION - `format_asset_check_executions(executions, as_json)` — table: STATUS | RUN_ID | TIMESTAMP | PARTITION ### 2. `dg api asset get-partition-status` (Step 1.3) Extend existing asset files: **Schema** — add to `dagster_dg_cli/api_layer/schemas/asset.py` - `DgApiPartitionStats` — num_materialized, num_failed, num_materializing, num_partitions **GraphQL adapter** — add to `dagster_dg_cli/api_layer/graphql_adapter/asset.py` - `ASSET_PARTITION_STATUS_QUERY` — query `assetNodeOrError(assetKey)` with `partitionStats { numMaterialized numPartitions numFailed numMaterializing }` - `get_asset_partition_status_via_graphql()` function **API class** — add to `dagster_dg_cli/api_layer/api/asset.py` - `get_partition_status(asset_key: str)` method on `DgApiAssetApi` **CLI** — add to `dagster_dg_cli/cli/api/asset.py` - `dg api asset get-partition-status ASSET_KEY [--json]` - Register in asset_group commands dict **Formatters** — add to `dagster_dg_cli/cli/api/formatters.py` - `format_partition_status(stats, as_json)` — detail view with materialized/failed/in-progress/total ### 3. Registration **`dagster_dg_cli/cli/api/cli_group.py`** — add `asset-check` entry: ```python from dagster_dg_cli.cli.api.asset_check import asset_check_group # in commands dict: "asset-check": asset_check_group, ``` ## Files to create - `dagster-oss/.../dagster_dg_cli/api_layer/schemas/asset_check.py` - `dagster-oss/.../dagster_dg_cli/api_layer/graphql_adapter/asset_check.py` - `dagster-oss/.../dagster_dg_cli/api_layer/api/asset_check.py` - `dagster-oss/.../dagster_dg_cli/cli/api/asset_check.py` ## Files to modify - `dagster-oss/.../dagster_dg_cli/api_layer/schemas/asset.py` — add `DgApiPartitionStats` - `dagster-oss/.../dagster_dg_cli/api_layer/graphql_adapter/asset.py` — add partition status query - `dagster-oss/.../dagster_dg_cli/api_layer/api/asset.py` — add `get_partition_status` method - `dagster-oss/.../dagster_dg_cli/cli/api/asset.py` — add `get-partition-status` command, register in group - `dagster-oss/.../dagster_dg_cli/cli/api/cli_group.py` — register `asset-check` group - `dagster-oss/.../dagster_dg_cli/cli/api/formatters.py` — add 3 formatter functions ## Key patterns to follow - Use `@dg_response_schema`, `@dg_api_options(deployment_scoped=True)`, `@cli_telemetry_wrapper` decorators - Use `DagsterPlusCliConfig.create_for_deployment()` for config - Use `create_dg_api_graphql_client()` for client - Use `handle_api_errors()` context manager - Pydantic BaseModel for schemas (not @record — per schemas/CLAUDE.md) - `--asset-key` as a required click option (not argument) for `asset-check` commands - Handle GraphQL union types (`AssetChecksOrError` can return `NeedsMigration`, `NeedsUserCodeUpgrade`, `NeedsAgentUpgrade`) ## GraphQL queries to use - **Asset checks**: `assetChecksOrError(assetKey: AssetKeyInput!)` on root query — returns `AssetChecks` with `checks` list - **Check executions**: `assetCheckExecutions(assetKey, checkName, limit, cursor)` on root query — returns list of `AssetCheckExecution` - **Partition stats**: `assetNodeOrError(assetKey)` → `partitionStats { numMaterialized numPartitions numFailed numMaterializing }` on `AssetNode` ## Verification 1. `just ruff` — formatting/linting 2. `just pyright dagster-oss/python_modules/libraries/dagster-dg-cli` — type checking 3. Manual test against a Dagster Plus deployment: - `dg api asset-check list --asset-key my/asset` - `dg api asset-check get-executions --asset-key my/asset --check-name my_check` - `dg api asset get-partition-status my/asset` - Verify both table and `--json` output </details> <!-- WARNING: Machine-generated. Manual edits may break erk tooling. --> <!-- erk:metadata-block:plan-header --> <details> <summary>plan-header</summary> ```yaml schema_version: '2' created_at: '2026-03-27T13:08:46.961224+00:00' created_by: OwenKephart plan_comment_id: null last_dispatched_run_id: null last_dispatched_node_id: null last_dispatched_at: null last_local_impl_at: '2026-03-27T20:16:14.922955+00:00' last_local_impl_event: ended last_local_impl_session: bca30886-68ef-4d5a-86a8-da0b723c5f33 last_local_impl_user: owen last_remote_impl_at: null last_remote_impl_run_id: null last_remote_impl_session_id: null branch_name: plnd/O22131-add-asset-check-par-03-27-1308 objective_issue: 22131 created_from_session: 00f71716-732a-4986-8bbd-911aca48d96e lifecycle_stage: impl worktree_name: erk-slot-02 last_session_branch: planned-pr-context/22132 last_session_id: bca30886-68ef-4d5a-86a8-da0b723c5f33 last_session_at: '2026-03-27T13:16:18.334090+00:00' last_session_source: local ``` </details> <!-- /erk:metadata-block:plan-header --> --- To replicate this PR locally, run: ``` erk pr teleport 22132 ``` Internal-RevId: 1ebfdadb9f73b5220e7109874d45c795125a5e6a
1 parent 679d84b commit 9d15a66

31 files changed

Lines changed: 1483 additions & 0 deletions

File tree

python_modules/libraries/dagster-dg-cli/dagster_dg_cli/api_layer/api/asset.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
get_asset_evaluations_via_graphql,
99
get_asset_events_via_graphql,
1010
get_asset_health_via_graphql,
11+
get_asset_partition_status_via_graphql,
1112
get_dg_plus_api_asset_via_graphql,
1213
list_dg_plus_api_assets_via_graphql,
1314
)
@@ -20,6 +21,7 @@
2021
DgApiAssetList,
2122
DgApiAssetStatus,
2223
DgApiEvaluationRecordList,
24+
DgApiPartitionStats,
2325
)
2426

2527

@@ -116,3 +118,8 @@ def get_evaluations(
116118
cursor=cursor,
117119
include_nodes=include_nodes,
118120
)
121+
122+
def get_partition_status(self, asset_key: str) -> "DgApiPartitionStats":
123+
"""Get partition materialization stats for an asset by slash-separated key."""
124+
asset_key_parts = asset_key.split("/")
125+
return get_asset_partition_status_via_graphql(self.client, asset_key_parts)
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
"""Asset check API implementation."""
2+
3+
from dataclasses import dataclass
4+
from typing import TYPE_CHECKING
5+
6+
from dagster_dg_cli.api_layer.graphql_adapter.asset_check import (
7+
get_asset_check_executions_via_graphql,
8+
list_asset_checks_via_graphql,
9+
)
10+
from dagster_dg_cli.utils.plus.gql_client import IGraphQLClient
11+
12+
if TYPE_CHECKING:
13+
from dagster_dg_cli.api_layer.schemas.asset_check import (
14+
DgApiAssetCheckExecutionList,
15+
DgApiAssetCheckList,
16+
)
17+
18+
19+
@dataclass(frozen=True)
20+
class DgApiAssetCheckApi:
21+
"""API for asset check operations."""
22+
23+
client: IGraphQLClient
24+
25+
def list_asset_checks(self, asset_key: str) -> "DgApiAssetCheckList":
26+
"""List asset checks for a given asset key."""
27+
return list_asset_checks_via_graphql(self.client, asset_key)
28+
29+
def get_check_executions(
30+
self,
31+
*,
32+
asset_key: str,
33+
check_name: str,
34+
limit: int = 25,
35+
cursor: str | None = None,
36+
) -> "DgApiAssetCheckExecutionList":
37+
"""Get execution history for an asset check."""
38+
return get_asset_check_executions_via_graphql(
39+
self.client,
40+
asset_key=asset_key,
41+
check_name=check_name,
42+
limit=limit,
43+
cursor=cursor,
44+
)

python_modules/libraries/dagster-dg-cli/dagster_dg_cli/api_layer/graphql_adapter/asset.py

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
DgApiEvaluationRecordList,
2020
DgApiPartitionDefinition,
2121
DgApiPartitionMapping,
22+
DgApiPartitionStats,
2223
)
2324
from dagster_dg_cli.utils.plus.gql_client import IGraphQLClient
2425

@@ -786,3 +787,55 @@ def get_asset_evaluations_via_graphql(
786787
)
787788

788789
return DgApiEvaluationRecordList(items=evaluations)
790+
791+
792+
# ---------------------------------------------------------------------------
793+
# Partition status
794+
# ---------------------------------------------------------------------------
795+
796+
ASSET_PARTITION_STATUS_QUERY = """
797+
query AssetPartitionStatusQuery($assetKey: AssetKeyInput!) {
798+
assetNodeOrError(assetKey: $assetKey) {
799+
__typename
800+
... on AssetNode {
801+
partitionStats {
802+
numMaterialized
803+
numPartitions
804+
numFailed
805+
numMaterializing
806+
}
807+
}
808+
... on AssetNotFoundError {
809+
message
810+
}
811+
}
812+
}
813+
"""
814+
815+
816+
def get_asset_partition_status_via_graphql(
817+
client: IGraphQLClient, asset_key_parts: list[str]
818+
) -> DgApiPartitionStats:
819+
"""Fetch partition materialization stats for an asset."""
820+
variables = {"assetKey": {"path": asset_key_parts}}
821+
result = client.execute(ASSET_PARTITION_STATUS_QUERY, variables)
822+
823+
asset_node = result.get("assetNodeOrError")
824+
if not asset_node:
825+
raise Exception("No asset data in GraphQL response")
826+
827+
typename = asset_node.get("__typename")
828+
if typename != "AssetNode":
829+
error_msg = asset_node.get("message", f"GraphQL error: {typename}")
830+
raise Exception(error_msg)
831+
832+
partition_stats = asset_node.get("partitionStats")
833+
if not partition_stats:
834+
raise Exception("Asset does not have partitions")
835+
836+
return DgApiPartitionStats(
837+
num_materialized=partition_stats["numMaterialized"],
838+
num_failed=partition_stats["numFailed"],
839+
num_materializing=partition_stats["numMaterializing"],
840+
num_partitions=partition_stats["numPartitions"],
841+
)
Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
"""GraphQL implementation for asset check operations."""
2+
3+
from typing import TYPE_CHECKING, Any
4+
5+
from dagster_dg_cli.utils.plus.gql_client import IGraphQLClient
6+
7+
if TYPE_CHECKING:
8+
from dagster_dg_cli.api_layer.schemas.asset_check import (
9+
DgApiAssetCheckExecutionList,
10+
DgApiAssetCheckList,
11+
)
12+
13+
ASSET_CHECKS_QUERY = """
14+
query AssetChecksQuery($assetKey: AssetKeyInput!) {
15+
assetChecksOrError(assetKey: $assetKey) {
16+
__typename
17+
... on AssetChecks {
18+
checks {
19+
name
20+
description
21+
blocking
22+
jobNames
23+
canExecuteIndividually
24+
assetKey {
25+
path
26+
}
27+
}
28+
}
29+
... on NeedsMigration {
30+
message
31+
}
32+
... on NeedsUserCodeUpgrade {
33+
message
34+
}
35+
... on NeedsAgentUpgrade {
36+
message
37+
}
38+
}
39+
}
40+
"""
41+
42+
ASSET_CHECK_EXECUTIONS_QUERY = """
43+
query AssetCheckExecutionsQuery(
44+
$assetKey: AssetKeyInput!,
45+
$checkName: String!,
46+
$limit: Int!,
47+
$cursor: String
48+
) {
49+
assetCheckExecutions(
50+
assetKey: $assetKey,
51+
checkName: $checkName,
52+
limit: $limit,
53+
cursor: $cursor
54+
) {
55+
id
56+
runId
57+
status
58+
timestamp
59+
stepKey
60+
evaluation {
61+
severity
62+
timestamp
63+
targetMaterialization {
64+
runId
65+
timestamp
66+
}
67+
metadataEntries {
68+
label
69+
}
70+
}
71+
}
72+
}
73+
"""
74+
75+
76+
def process_asset_checks_response(
77+
graphql_response: dict[str, Any], asset_key: str
78+
) -> "DgApiAssetCheckList":
79+
"""Process GraphQL response into DgApiAssetCheckList."""
80+
from dagster_dg_cli.api_layer.schemas.asset_check import DgApiAssetCheck, DgApiAssetCheckList
81+
82+
checks_result = graphql_response.get("assetChecksOrError")
83+
if not checks_result:
84+
raise Exception("No asset checks data in GraphQL response")
85+
86+
typename = checks_result.get("__typename")
87+
if typename != "AssetChecks":
88+
error_msg = checks_result.get("message", f"GraphQL error: {typename}")
89+
raise Exception(error_msg)
90+
91+
checks_data = checks_result.get("checks", [])
92+
93+
checks = []
94+
for c in checks_data:
95+
check_asset_key = "/".join(c.get("assetKey", {}).get("path", []))
96+
checks.append(
97+
DgApiAssetCheck(
98+
name=c["name"],
99+
asset_key=check_asset_key or asset_key,
100+
description=c.get("description"),
101+
blocking=c.get("blocking", False),
102+
job_names=c.get("jobNames", []),
103+
can_execute_individually=c.get("canExecuteIndividually"),
104+
)
105+
)
106+
107+
return DgApiAssetCheckList(items=checks)
108+
109+
110+
def process_asset_check_executions_response(
111+
graphql_response: dict[str, Any], asset_key: str, check_name: str
112+
) -> "DgApiAssetCheckExecutionList":
113+
"""Process GraphQL response into DgApiAssetCheckExecutionList."""
114+
from dagster_dg_cli.api_layer.schemas.asset_check import (
115+
DgApiAssetCheckExecution,
116+
DgApiAssetCheckExecutionList,
117+
DgApiAssetCheckExecutionStatus,
118+
)
119+
120+
executions_data = graphql_response.get("assetCheckExecutions", [])
121+
122+
executions = []
123+
for e in executions_data:
124+
executions.append(
125+
DgApiAssetCheckExecution(
126+
id=e["id"],
127+
run_id=e["runId"],
128+
status=DgApiAssetCheckExecutionStatus(e["status"]),
129+
timestamp=float(e["timestamp"]),
130+
step_key=e.get("stepKey"),
131+
check_name=check_name,
132+
asset_key=asset_key,
133+
)
134+
)
135+
136+
return DgApiAssetCheckExecutionList(items=executions)
137+
138+
139+
def list_asset_checks_via_graphql(client: IGraphQLClient, asset_key: str) -> "DgApiAssetCheckList":
140+
"""Fetch asset checks using GraphQL."""
141+
asset_key_parts = asset_key.split("/")
142+
variables = {"assetKey": {"path": asset_key_parts}}
143+
result = client.execute(ASSET_CHECKS_QUERY, variables)
144+
return process_asset_checks_response(result, asset_key)
145+
146+
147+
def get_asset_check_executions_via_graphql(
148+
client: IGraphQLClient,
149+
*,
150+
asset_key: str,
151+
check_name: str,
152+
limit: int = 25,
153+
cursor: str | None = None,
154+
) -> "DgApiAssetCheckExecutionList":
155+
"""Fetch asset check executions using GraphQL."""
156+
asset_key_parts = asset_key.split("/")
157+
variables: dict[str, Any] = {
158+
"assetKey": {"path": asset_key_parts},
159+
"checkName": check_name,
160+
"limit": limit,
161+
}
162+
if cursor:
163+
variables["cursor"] = cursor
164+
165+
result = client.execute(ASSET_CHECK_EXECUTIONS_QUERY, variables)
166+
return process_asset_check_executions_response(result, asset_key, check_name)

python_modules/libraries/dagster-dg-cli/dagster_dg_cli/api_layer/schemas/asset.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -181,3 +181,12 @@ class DgApiEvaluationRecordList(BaseModel):
181181
"""GET /api/assets/{key}/evaluations response."""
182182

183183
items: list[DgApiEvaluationRecord]
184+
185+
186+
class DgApiPartitionStats(BaseModel):
187+
"""Partition materialization statistics for an asset."""
188+
189+
num_materialized: int
190+
num_failed: int
191+
num_materializing: int
192+
num_partitions: int
Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
"""Asset check schema definitions."""
2+
3+
from enum import Enum
4+
5+
from pydantic import BaseModel
6+
7+
8+
class DgApiAssetCheckExecutionStatus(str, Enum):
9+
"""Asset check execution status."""
10+
11+
IN_PROGRESS = "IN_PROGRESS"
12+
SUCCEEDED = "SUCCEEDED"
13+
FAILED = "FAILED"
14+
SKIPPED = "SKIPPED"
15+
EXECUTION_FAILED = "EXECUTION_FAILED"
16+
17+
18+
class DgApiAssetCheck(BaseModel):
19+
"""Single asset check metadata model."""
20+
21+
name: str
22+
asset_key: str
23+
description: str | None = None
24+
blocking: bool = False
25+
job_names: list[str] = []
26+
can_execute_individually: str | None = None
27+
28+
29+
class DgApiAssetCheckList(BaseModel):
30+
"""List of asset checks response."""
31+
32+
items: list[DgApiAssetCheck]
33+
34+
35+
class DgApiAssetCheckExecution(BaseModel):
36+
"""Single asset check execution record."""
37+
38+
id: str
39+
run_id: str
40+
status: DgApiAssetCheckExecutionStatus
41+
timestamp: float
42+
partition: str | None = None
43+
step_key: str | None = None
44+
check_name: str
45+
asset_key: str
46+
47+
48+
class DgApiAssetCheckExecutionList(BaseModel):
49+
"""List of asset check executions response."""
50+
51+
items: list[DgApiAssetCheckExecution]
52+
cursor: str | None = None
53+
has_more: bool = False

0 commit comments

Comments
 (0)