Skip to content

Commit 0efd950

Browse files
srousseysroussey
authored
Merge pull request #719 from workglow-dev/claude/testing-structure-improvements-i7p0qq
Reorganize Tests and contracts
2 parents b081e9a + 858c461 commit 0efd950

444 files changed

Lines changed: 2258 additions & 1440 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.claude/CLAUDE.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -301,10 +301,35 @@ class TestTask extends Task<TestInput, TestOutput> {
301301

302302
```sh
303303
bun scripts/test.ts [--all] [kinds...] [sections...] [runners...] [options]
304+
bun scripts/test.ts --changed [base] # only packages affected since base (default origin/main)
304305
```
305306

306307
When making code changes, run the tests on that section only, and pass vitest only. Otherwise tests are very slow. For example, if you are making changes to the McpServer, run `bun scripts/test.ts mcp vitest`.
307308

309+
Sections are **discovered**, never enumerated — every directory holding tests maps to a
310+
section, and `--check-sections` fails if any test file is unreachable by section+kind
311+
selection. Note that `packages/test/src/test/task-graph*/` belongs to section `graph`,
312+
not `task-graph`; `task-graph` selects the package's own co-located `__tests__`.
313+
314+
`--changed` delegates package selection to Turbo (`turbo run test --filter=...[base]`),
315+
so a change also runs the tests of everything that depends on it. Only workspaces with a
316+
`test` script participate; tooling tests outside any workspace (`scripts/`) are not
317+
covered by that mode — run them with `bun scripts/test.ts scripts`.
318+
319+
### Vitest projects
320+
321+
The root `vitest.config.ts` defines one **project** per workspace that holds tests, and
322+
the project list is derived from the same discovery the runner uses rather than written
323+
out by hand. `vitest run --project task-graph` runs just that package; each package's own
324+
`test` script is `vitest run --config ../../vitest.config.ts --project <name>`, which is
325+
what Turbo invokes.
326+
327+
Anything path-shaped in the shared project options must be **absolute** — project roots
328+
differ, so a relative `setupFiles` or `typecheck.tsconfig` would resolve against each
329+
package and silently fail to load. `testDiscovery.test.ts` reads the real config and
330+
fails if any discovered test file falls outside every project root: such a file does not
331+
error or warn, it simply stops running.
332+
308333
### Developing without building
309334

310335
`bun run use-source` (or `./scripts/bunsrc-workspace.ts source`) makes every package resolve to its source files instead of its built files, so you can develop without rebuilding. It does **not** touch `package.json`: `exports` keeps pointing at `./dist/*`, and the script writes tiny re-export stubs into each package's (gitignored) `dist` folder — `dist/node.js` becomes `export * from "../src/node.ts"`, `dist/node.d.ts` the declaration equivalent. Source mode therefore leaves `git status` clean and there is nothing to revert before committing.

‎.github/workflows/nightly-bun-parity.yml‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
name: Nightly Bun runtime parity
2+
3+
# Bun is a supported local runner and occasionally surfaces bugs vitest does not,
4+
# but it is not worth gating every PR on: running the whole suite under Bun spends
5+
# most of its time re-verifying vitest's own semantics on a second runner.
6+
#
7+
# So it runs on a schedule instead, and on demand via workflow_dispatch — the
8+
# latter matters as much as the cron, because it makes "run Bun now, without a
9+
# local checkout" a first-class action.
10+
#
11+
# Failures here are informational. They never block a merge. Triage note: a
12+
# Bun-only failure in a file using `vi.*` may be a gap in Bun's vitest
13+
# compatibility shim rather than a product bug — pin such a file to vitest or
14+
# rewrite it off the mocking API, but do not change product code to satisfy the
15+
# shim.
16+
17+
permissions:
18+
contents: read
19+
20+
on:
21+
schedule:
22+
- cron: "43 6 * * *"
23+
workflow_dispatch:
24+
25+
env:
26+
CI: "true"
27+
DO_NOT_TRACK: "1"
28+
TURBO_TELEMETRY_DISABLED: "1"
29+
30+
jobs:
31+
bun-parity:
32+
runs-on: ubuntu-latest
33+
timeout-minutes: 45
34+
strategy:
35+
fail-fast: false
36+
matrix:
37+
kind: [unit, integration]
38+
steps:
39+
- uses: actions/checkout@v6
40+
- uses: actions/setup-node@v6
41+
with:
42+
node-version: 24
43+
- uses: oven-sh/setup-bun@v2
44+
with:
45+
bun-version: latest
46+
- run: bun i
47+
- run: bun run build
48+
# Excluded on purpose: the live tier (model downloads, paid APIs) belongs
49+
# to its own schedule, and `browser` needs Playwright binaries this job
50+
# does not install. Everything else is covered by subtraction, so a new
51+
# section joins this run without anyone remembering to add it.
52+
- name: Run ${{ matrix.kind }} tests under Bun
53+
env:
54+
WORKGLOW_SECRETS_PASSPHRASE: ${{ secrets.WORKGLOW_SECRETS_PASSPHRASE }}
55+
run: bun scripts/test.ts bun ${{ matrix.kind }} --except rag,browser,provider-hft,provider-nodellama,provider-api,provider-cactus

‎.github/workflows/test.yml‎

Lines changed: 23 additions & 142 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,27 @@ jobs:
3535
# growth with: bun run typecheck:budget --update
3636
- name: Typecheck budget guard
3737
run: bun run typecheck:budget
38+
# Co-located `__tests__` are excluded from each package's build program so
39+
# they never reach `dist`, which also takes them out of every tsc run.
40+
# `tsconfig.test.json` is the program that still checks them; without this
41+
# step a type error in a test file is invisible until someone runs it.
42+
# Must follow the budget guard, which is what emits the `dist/*.d.ts` these
43+
# programs resolve their own package against.
44+
- name: Typecheck co-located tests
45+
run: bun run typecheck:tests
46+
47+
test-discovery:
48+
runs-on: ubuntu-latest
49+
steps:
50+
- uses: actions/checkout@v6
51+
- uses: oven-sh/setup-bun@v2
52+
with:
53+
bun-version: latest
54+
- run: bun i
55+
# Fails if any test file is unreachable by section+kind selection. Every
56+
# other job passes a kind, so an unreachable file silently never runs.
57+
- name: Test discovery guard
58+
run: bun scripts/test.ts --check-sections
3859

3960
build:
4061
runs-on: ubuntu-latest
@@ -58,141 +79,6 @@ jobs:
5879
examples/*/dist/**
5980
retention-days: 1
6081

61-
# test-bun-unit:
62-
# runs-on: ubuntu-latest
63-
# needs: build
64-
# steps:
65-
# - uses: actions/checkout@v6
66-
# - uses: actions/setup-node@v6
67-
# with:
68-
# node-version: 24
69-
# - uses: oven-sh/setup-bun@v2
70-
# with:
71-
# bun-version: latest
72-
# - run: bun i
73-
# - name: Download build artifacts
74-
# uses: actions/download-artifact@v8
75-
# with:
76-
# name: build-output
77-
# path: .
78-
# - name: Run unit tests via bun
79-
# run: bun run test:bun:unit
80-
81-
# test-bun-integration:
82-
# runs-on: ubuntu-latest
83-
# needs: build
84-
# steps:
85-
# - uses: actions/checkout@v6
86-
# - uses: actions/setup-node@v6
87-
# with:
88-
# node-version: 24
89-
# - uses: oven-sh/setup-bun@v2
90-
# with:
91-
# bun-version: latest
92-
# - run: bun i
93-
# - name: Download build artifacts
94-
# uses: actions/download-artifact@v8
95-
# with:
96-
# name: build-output
97-
# path: .
98-
# - name: Run integration tests via bun
99-
# env:
100-
# WORKGLOW_SECRETS_PASSPHRASE: ${{ secrets.WORKGLOW_SECRETS_PASSPHRASE }}
101-
# run: bun run test:bun:integration
102-
103-
# test-bun-rag:
104-
# runs-on: ubuntu-latest
105-
# needs: build
106-
# timeout-minutes: 25
107-
# steps:
108-
# - uses: actions/checkout@v6
109-
# - uses: actions/setup-node@v6
110-
# with:
111-
# node-version: 24
112-
# - uses: oven-sh/setup-bun@v2
113-
# with:
114-
# bun-version: latest
115-
# - run: bun i
116-
# - name: Cache ONNX / Hugging Face model downloads
117-
# uses: actions/cache@v4
118-
# with:
119-
# path: |
120-
# ~/.cache/huggingface
121-
# key: libs-hf-models-${{ runner.os }}-${{ hashFiles('bun.lock') }}
122-
# restore-keys: |
123-
# libs-hf-models-${{ runner.os }}-
124-
# - name: Download build artifacts
125-
# uses: actions/download-artifact@v8
126-
# with:
127-
# name: build-output
128-
# path: .
129-
# - name: Run rag tests via bun
130-
# env:
131-
# WORKGLOW_SECRETS_PASSPHRASE: ${{ secrets.WORKGLOW_SECRETS_PASSPHRASE }}
132-
# run: bun run test:bun:rag
133-
134-
# test-bun-ai-provider-hft:
135-
# runs-on: ubuntu-latest
136-
# needs: build
137-
# steps:
138-
# - uses: actions/checkout@v6
139-
# - uses: actions/setup-node@v6
140-
# with:
141-
# node-version: 24
142-
# - uses: oven-sh/setup-bun@v2
143-
# with:
144-
# bun-version: latest
145-
# - run: bun i
146-
# - name: Download build artifacts
147-
# uses: actions/download-artifact@v8
148-
# with:
149-
# name: build-output
150-
# path: .
151-
# - name: Run HuggingFace Transformers provider tests via bun
152-
# run: bun run test:bun:ai-provider-hft
153-
154-
# test-bun-ai-provider-nodellama:
155-
# runs-on: ubuntu-latest
156-
# needs: build
157-
# steps:
158-
# - uses: actions/checkout@v6
159-
# - uses: actions/setup-node@v6
160-
# with:
161-
# node-version: 24
162-
# - uses: oven-sh/setup-bun@v2
163-
# with:
164-
# bun-version: latest
165-
# - run: bun i
166-
# - name: Download build artifacts
167-
# uses: actions/download-artifact@v8
168-
# with:
169-
# name: build-output
170-
# path: .
171-
# - name: Run LlamaCpp provider tests via bun
172-
# run: bun run test:bun:ai-provider-nodellama
173-
174-
# test-bun-ai-provider-api:
175-
# runs-on: ubuntu-latest
176-
# needs: build
177-
# steps:
178-
# - uses: actions/checkout@v6
179-
# - uses: actions/setup-node@v6
180-
# with:
181-
# node-version: 24
182-
# - uses: oven-sh/setup-bun@v2
183-
# with:
184-
# bun-version: latest
185-
# - run: bun i
186-
# - name: Download build artifacts
187-
# uses: actions/download-artifact@v8
188-
# with:
189-
# name: build-output
190-
# path: .
191-
# - name: Run API provider tests via bun
192-
# env:
193-
# WORKGLOW_SECRETS_PASSPHRASE: ${{ secrets.WORKGLOW_SECRETS_PASSPHRASE }}
194-
# run: bun run test:bun:ai-provider-api
195-
19682
test-vitest-unit:
19783
runs-on: ubuntu-latest
19884
needs: build
@@ -455,13 +341,8 @@ jobs:
455341

456342
cleanup:
457343
runs-on: ubuntu-latest
458-
needs: [
459-
# test-bun-unit,
460-
# test-bun-integration,
461-
# test-bun-rag,
462-
# test-bun-ai-provider-hft,
463-
# test-bun-ai-provider-nodellama,
464-
# test-bun-ai-provider-api,
344+
needs:
345+
[
465346
test-vitest-unit,
466347
test-vitest-integration,
467348
test-vitest-rag,

‎examples/cli/package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323
"build-js-worker-hft": "bun build --target=node --packages=external --outdir ./dist ./src/worker_hft.ts",
2424
"build-lib": "bun build --target=bun --packages=external --outdir ./dist ./src/lib.ts",
2525
"build-types": "rm -f tsconfig.tsbuildinfo && tsgo",
26-
"test": "vitest run",
26+
"test": "vitest run --config ../../vitest.config.ts --project cli",
2727
"test:watch": "vitest"
2828
},
2929
"bin": "./dist/workglow.js",

‎examples/eval/package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
"build-js": "bun build --target=bun --packages=external --outdir ./dist ./src/workglow-eval.ts",
2222
"build-js-worker-hft": "bun build --target=node --packages=external --outdir ./dist ./src/worker_hft.ts",
2323
"build-types": "rm -f tsconfig.tsbuildinfo && tsgo",
24-
"test": "vitest run",
24+
"test": "vitest run --config ../../vitest.config.ts --project eval",
2525
"test:watch": "vitest",
2626
"build-js-worker-llamacpp": "bun build --target=node --packages=external --outdir ./dist ./src/worker_llamacpp.ts"
2727
},

‎package.json‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,21 +26,22 @@
2626
"build:js": "turbo run build-js --concurrency=15",
2727
"build:types": "turbo run build-types --concurrency=15",
2828
"typecheck:budget": "bun scripts/typecheck-budget.ts",
29+
"typecheck:tests": "for f in packages/*/tsconfig.test.json; do echo \"typecheck $f\" && npx tsgo -p \"$f\" || exit 1; done",
2930
"clean": "rm -rf node_modules packages/*/node_modules packages/*/*tsbuildinfo packages/*/dist packages/*/src/**/*\\.d\\.ts packages/*/src/**/*\\.map integrations/*/node_modules integrations/*/dist integrations/*/src/**/*\\.d\\.ts integrations/*/src/**/*\\.map examples/*/node_modules examples/*/dist examples/*/src/**/*\\.d\\.ts examples/*/src/**/*\\.map .turbo */*/.turbo",
3031
"dev": "turbo run dev --concurrency=15",
3132
"docs": "typedoc",
3233
"format": "eslint --fix && prettier \"{packages,providers,examples}/*/src/**/*.{js,ts,tsx,json}\" --check --write",
3334
"build:release": "turbo run build-js build-types --concurrency=15",
3435
"test": "bun scripts/test.ts",
3536
"test:bun:unit": "bun scripts/test.ts bun unit",
36-
"test:bun:integration": "bun scripts/test.ts bun integration graph task storage queue util mcp",
37+
"test:bun:integration": "bun scripts/test.ts bun integration --except rag,browser,provider-hft,provider-nodellama,provider-api,provider-cactus",
3738
"test:bun:rag": "bun scripts/test.ts bun integration rag",
3839
"test:bun:ai-provider": "bun scripts/test.ts bun integration ai provider",
3940
"test:bun:ai-provider-hft": "bun scripts/test.ts bun integration provider-hft",
4041
"test:bun:ai-provider-nodellama": "bun scripts/test.ts bun integration provider-nodellama",
4142
"test:bun:ai-provider-api": "bun scripts/test.ts bun integration ai provider-api",
4243
"test:vitest:unit": "bun scripts/test.ts vitest unit",
43-
"test:vitest:integration": "bun scripts/test.ts vitest integration graph task storage queue util mcp",
44+
"test:vitest:integration": "bun scripts/test.ts vitest integration --except rag,browser,provider-hft,provider-nodellama,provider-api,provider-cactus",
4445
"test:vitest:rag": "bun scripts/test.ts vitest integration rag",
4546
"test:vitest:ai-provider": "bun scripts/test.ts vitest integration ai provider",
4647
"test:vitest:ai-provider-hft": "bun scripts/test.ts vitest integration provider-hft",
@@ -156,4 +157,4 @@
156157
"adm-zip": "^0.6.0",
157158
"shell-quote": "^1.10.0"
158159
}
159-
}
160+
}

‎packages/ai/package.json‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,21 +15,24 @@
1515
"homepage": "https://workglow.dev",
1616
"scripts": {
1717
"watch": "concurrently -c 'auto' 'bun:watch-*'",
18-
"watch-js": "concurrently -c 'auto' -n 'browser,node,worker,provider-utils' 'bun run watch-browser' 'bun run watch-node' 'bun run watch-worker' 'bun run watch-provider-utils'",
18+
"watch-js": "concurrently -c 'auto' -n 'browser,node,worker,provider-utils,test' 'bun run watch-browser' 'bun run watch-node' 'bun run watch-worker' 'bun run watch-provider-utils' 'bun run watch-test'",
1919
"watch-browser": "bun build --watch --no-clear-screen --target=browser --sourcemap=external --packages=external --outdir ./dist ./src/browser.ts",
2020
"watch-node": "bun build --watch --no-clear-screen --target=node --sourcemap=external --packages=external --outdir ./dist ./src/node.ts",
2121
"watch-worker": "bun build --watch --no-clear-screen --target=browser --sourcemap=external --packages=external --outdir ./dist ./src/worker.ts",
2222
"watch-provider-utils": "bun build --watch --no-clear-screen --sourcemap=external --packages=external --outdir ./dist ./src/provider-utils.ts",
2323
"watch-types": "tsc --watch --preserveWatchOutput",
24-
"build-package": "concurrently -c 'auto' -n 'browser,node,worker,provider-utils,types' 'bun run build-browser' 'bun run build-node' 'bun run build-worker' 'bun run build-provider-utils' 'bun run build-types'",
25-
"build-js": "concurrently -m 12 --timings -c 'auto' -n 'browser,node,worker,provider-utils' 'bun run build-browser' 'bun run build-node' 'bun run build-worker' 'bun run build-provider-utils'",
24+
"build-package": "concurrently -c 'auto' -n 'browser,node,worker,provider-utils,test,types' 'bun run build-browser' 'bun run build-node' 'bun run build-worker' 'bun run build-provider-utils' 'bun run build-test' 'bun run build-types'",
25+
"build-js": "concurrently -m 12 --timings -c 'auto' -n 'browser,node,worker,provider-utils,test' 'bun run build-browser' 'bun run build-node' 'bun run build-worker' 'bun run build-provider-utils' 'bun run build-test'",
2626
"build-clean": "rm -fr dist/* tsconfig.tsbuildinfo",
2727
"build-browser": "bun build --target=browser --sourcemap=external --packages=external --outdir ./dist ./src/browser.ts",
2828
"build-node": "bun build --target=node --sourcemap=external --packages=external --outdir ./dist ./src/node.ts",
2929
"build-worker": "bun build --target=browser --sourcemap=external --packages=external --outdir ./dist ./src/worker.ts",
3030
"build-provider-utils": "bun build --sourcemap=external --packages=external --outdir ./dist ./src/provider-utils.ts",
3131
"build-types": "rm -f tsconfig.tsbuildinfo && tsgo",
32-
"lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0"
32+
"lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0",
33+
"build-test": "bun build --target=node --sourcemap=external --packages=external --outdir ./dist ./src/test-entry.ts",
34+
"test": "vitest run --config ../../vitest.config.ts --project ai",
35+
"watch-test": "bun build --watch --no-clear-screen --target=node --sourcemap=external --packages=external --outdir ./dist ./src/test-entry.ts"
3336
},
3437
"exports": {
3538
".": {
@@ -51,6 +54,10 @@
5154
"./provider-utils": {
5255
"types": "./dist/provider-utils.d.ts",
5356
"import": "./dist/provider-utils.js"
57+
},
58+
"./test": {
59+
"types": "./dist/test-entry.d.ts",
60+
"import": "./dist/test-entry.js"
5461
}
5562
},
5663
"sideEffects": false,

packages/test/src/test/ai/StreamEventAccumulator.test.ts renamed to packages/ai/src/capability/__tests__/StreamEventAccumulator.test.ts

File renamed without changes.

packages/test/src/test/ai/accumulatingEmit.test.ts renamed to packages/ai/src/capability/__tests__/accumulatingEmit.test.ts

File renamed without changes.

packages/test/src/test/ai/collectStream.test.ts renamed to packages/ai/src/capability/__tests__/collectStream.test.ts

File renamed without changes.

0 commit comments

Comments
 (0)