Skip to content

Commit b8c776f

Browse files
SpencerWhitehead7Dagster Devtools
authored andcommitted
maint: centralize, standardize graphql codegen just commands as python scripts (#25503)
## Summary & Motivation Make graphql based codegen more consistent and understandable. Root level just command to regenerate all graphql code, across internal and oss. |-> dagster-cloud just command to regenerate all graphql code in internal |-> dagster-oss jus command to regenerate all graphql code in oss commands generate both FE and BE code in the order necessary for both to be fully updated (FE used to generate a schema, which BE then relied upon, which seems super backward, but whatever). There are still two strange things left with this, which I couldn't think of a good solution for. 1 BE codegen relies on FE codegen - the PR handles this by just making sure they run in the right order. I think ideally there'd be a step that created the schema.graphql files for app-cloud and admin-cloud / oss in some kind of central, intuitively shared location, which BE and FE specific codegen steps would both reference, but I wasn't sure about the FE codegen tools reaching outside their directories and increasing the blast radius. Maybe later. 2 OSS codegen relies on Internal codegen - I think this is just intractable. dagster-rest-resources, an OSS project, relies on hitting internal graphql resolvers, which means its codegenerated client needs to read the internal schema.graphql. dagster-rest-resources needs access to those resolvers to support functionality in dg-cli, which is also an OSS project. The root is the fact that OSS stuff truly does rely on Internal stuff, and that showing up in the codegen is just a manifestation of that root weirdness. This PR does handle that whole situation a little more elegantly than the status quo. It will skip that step if it cannot successfully resolve the internal schema, instead of trying to run against a file that might not exist. The actual substantial, non codegen changes in the diff are not so big ## Test Plan delete all the codegen stuff locally, regenerate, see if anything changes or fails to regenerate ci Internal-RevId: 4deddedb539cdade9a4121030abcfa318fe52a14
1 parent f0b753b commit b8c776f

13 files changed

Lines changed: 770 additions & 158 deletions

File tree

.claude/ui_workflow.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@
77
1. **[CONDITIONAL] Regenerate GraphQL types if schema changed**:
88

99
```bash
10-
cd $DAGSTER_GIT_REPO_DIR/js_modules
11-
make generate-graphql
10+
cd $DAGSTER_GIT_REPO_DIR
11+
just generate-graphql
1212
```
1313

1414
**IMPORTANT**: This MUST complete BEFORE running `yarn tsgo` or `yarn lint`, as it updates TypeScript type definitions.
@@ -84,7 +84,7 @@ This ensures changes to shared UI packages don't break the Cloud application.
8484
- **Location**: Must run from `js_modules/`
8585
- **Command**: `yarn jest` (not `make jest`)
8686

87-
### GraphQL Generation (`make generate-graphql`)
87+
### GraphQL Generation (`just generate-graphql`)
8888

8989
- **Purpose**: Regenerate GraphQL types from schema
9090
- **When**: After GraphQL schema changes in the backend
@@ -156,7 +156,7 @@ cd $DAGSTER_GIT_REPO_DIR/js_modules
156156
# CRITICAL ORDER: generate-graphql MUST complete BEFORE yarn tsgo/yarn lint
157157

158158
# 1. Regenerate types (MUST complete first)
159-
make generate-graphql
159+
just generate-graphql
160160

161161
# 2. Type checking (needs updated GraphQL types)
162162
yarn tsgo
@@ -168,7 +168,7 @@ yarn lint
168168
yarn jest
169169
```
170170

171-
**Why this order matters**: `make generate-graphql` updates TypeScript type definitions based on the backend schema. Running `yarn tsgo` or `yarn lint` before these types are updated will result in false type errors.
171+
**Why this order matters**: `just generate-graphql` updates TypeScript type definitions based on the backend schema. Running `yarn tsgo` or `yarn lint` before these types are updated will result in false type errors.
172172

173173
### UI Components Package Changes
174174

CLAUDE.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,15 +31,15 @@ tox -e py39-pytest # Test in isolated environment
3131

3232
# UI Verification (run from js_modules/ when done with edits)
3333
cd js_modules
34-
make generate-graphql # Regenerate GraphQL types (if schema changed; must run before tsgo/lint)
34+
just generate-graphql # Regenerate GraphQL types (if schema changed; must run before tsgo/lint)
3535
yarn tsgo # TypeScript checking
3636
yarn lint # ESLint checking
3737
yarn jest # Jest tests
3838
yarn build # Verify production build (ui-components changes only)
3939

4040
# Development
4141
make rebuild_ui # Rebuild React UI after changes
42-
make graphql_codegen # Regenerate GraphQL schema and codegen clients
42+
just generate-graphql # Regenerate GraphQL schema and codegen clients
4343
make sanity_check # Check for non-editable installs
4444
```
4545

Makefile

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -58,15 +58,6 @@ install_dev_python_modules_verbose:
5858
install_dev_python_modules_verbose_m1:
5959
python scripts/install_dev_python_modules.py --include-prebuilt-grpcio-wheel
6060

61-
graphql_codegen_js:
62-
cd js_modules; make generate-graphql; make generate-perms
63-
64-
graphql_codegen_py:
65-
cd python_modules/libraries/dagster-rest-resources; uv run --active --no-project ariadne-codegen
66-
67-
graphql_codegen:
68-
make graphql_codegen_js;
69-
make graphql_codegen_py;
7061

7162
sanity_check:
7263
#NOTE: fails on nonPOSIX-compliant shells (e.g. CMD, powershell)

js_modules/Makefile

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,6 @@
11
dev_webapp:
22
NEXT_PUBLIC_BACKEND_ORIGIN="http://127.0.0.1:3333" yarn workspace @dagster-io/app-oss start
33

4-
generate-graphql:
5-
yarn workspace @dagster-io/ui-core generate-graphql
6-
7-
generate-perms:
8-
yarn workspace @dagster-io/ui-core generate-perms
94

105
ts:
116
yarn workspace @dagster-io/app-oss ts && yarn workspace @dagster-io/ui-core ts && yarn workspace @dagster-io/ui-components ts

js_modules/tox.ini

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,7 @@ commands =
3030
corepack enable
3131
!windows: /bin/bash -c '! uv pip list --exclude-editable | grep -e dagster'
3232
yarn install
33-
yarn workspace @dagster-io/ui-core generate-graphql
34-
yarn workspace @dagster-io/ui-core generate-perms
33+
python ../scripts/generate_graphql.py
3534
yarn workspace @dagster-io/ui-core tsgo
3635
yarn workspace @dagster-io/ui-core tsgo:workers
3736
yarn workspace @dagster-io/ui-core lint:ci

js_modules/ui-core/package.json

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,6 @@
1515
"tsgo": "tsgo -p .",
1616
"ts:workers": "tsc -p ./tsconfig.workers.json",
1717
"tsgo:workers": "tsgo -p ./tsconfig.workers.json",
18-
"generate-graphql": "ts-node -O '{\"module\": \"commonjs\"}' ./src/scripts/generateGraphQLTypes.ts",
19-
"generate-perms": "ts-node -O '{\"module\": \"commonjs\"}' ./src/scripts/generatePermissions.ts",
2018
"generate-asset-selection": "ts-node -O '{\"module\": \"commonjs\"}' ./src/scripts/generateAssetSelection.ts",
2119
"generate-selection-autocomplete": "ts-node -O '{\"module\": \"commonjs\"}' ./src/scripts/generateSelection.ts",
2220
"generate-run-selection": "ts-node -O '{\"module\": \"commonjs\"}' ./src/scripts/generateRunSelection.ts",

js_modules/ui-core/src/graphql/permissions.json

Lines changed: 22 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)