Skip to content

Commit 83fdb6b

Browse files
nhortonclaudegithub-actions[bot]
authored
Add automated CI test for Claude code (#44)
* Fix CLA signatures file format Replace Markdown with JSON array - CLA Assistant action expects JSON format for storing signatures. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Fix CLA signatures JSON structure CLA Assistant expects { "signedContributors": [] } format, not a plain array. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Remove manually created signatures file Per CLA Assistant docs: "You do not need to create this file manually. Our workflow will create the signature file if it does not already exist. Manually creating this file will cause the workflow to fail." Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Creating file for storing CLA Signatures * @nhorton has signed the CLA in #27 * @tylerwillis has signed the CLA in #31 * Add automated CI test for Claude Code integration Add a deterministic 'fruits' test job and comprehensive CI tests to validate that deepwork-generated commands work correctly with Claude Code. Changes: - Add fruits job fixture (identify + classify steps) for CI testing - Add integration tests for fruits workflow (8 tests) - Add e2e tests for Claude Code execution (3 tests, skipped without API key) - Add GitHub Actions workflow for automated testing: - validate-generation: Always runs, tests command generation - claude-code-e2e: Runs with ANTHROPIC_API_KEY, tests actual execution The fruits job is designed to be deterministic: - Input: comma-separated list of items (e.g., "apple, car, banana") - Step 1: Identify which items are fruits - Step 2: Classify fruits by category (citrus, tropical, etc.) * Fix CI workflow: use deepwork install with --path, add concurrency rules - Add concurrency rules to ensure only one instance runs per PR - Fix test to use 'deepwork install --platform claude --path test_project' - Create .claude directory before install for platform detection - Run commands from repo root with --path flag instead of cd'ing --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
1 parent c9bac58 commit 83fdb6b

8 files changed

Lines changed: 944 additions & 25 deletions

File tree

‎.github/workflows/claude-code-test.yml‎

Lines changed: 222 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,222 @@
1+
name: Claude Code Integration Test
2+
3+
on:
4+
# Manual trigger for testing
5+
workflow_dispatch:
6+
inputs:
7+
debug:
8+
description: 'Enable debug logging'
9+
required: false
10+
default: 'false'
11+
type: boolean
12+
# Run on PRs that modify core code
13+
pull_request:
14+
branches: ["*"]
15+
paths:
16+
- 'src/deepwork/**'
17+
- 'tests/**'
18+
- '.github/workflows/claude-code-test.yml'
19+
# Scheduled run for continuous validation
20+
schedule:
21+
- cron: '0 6 * * 1' # Weekly on Monday at 6 AM UTC
22+
23+
# Ensure only one instance runs at a time per PR/branch
24+
concurrency:
25+
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
26+
cancel-in-progress: true
27+
28+
jobs:
29+
# Job 1: Validate command generation (always runs)
30+
validate-generation:
31+
runs-on: ubuntu-latest
32+
steps:
33+
- uses: actions/checkout@v4
34+
35+
- name: Install uv
36+
uses: astral-sh/setup-uv@v4
37+
with:
38+
version: "latest"
39+
40+
- name: Set up Python
41+
uses: actions/setup-python@v5
42+
with:
43+
python-version: "3.11"
44+
45+
- name: Install dependencies
46+
run: uv sync --extra dev
47+
48+
- name: Run fruits workflow tests
49+
run: uv run pytest tests/integration/test_fruits_workflow.py -v
50+
51+
- name: Generate commands and validate structure
52+
run: |
53+
# Create a test environment
54+
mkdir -p test_project/.deepwork/jobs
55+
mkdir -p test_project/.claude # Required for platform detection
56+
cp -r tests/fixtures/jobs/fruits test_project/.deepwork/jobs/
57+
58+
# Set up git repo in test project
59+
cd test_project
60+
git init
61+
git config user.email "test@test.com"
62+
git config user.name "Test"
63+
echo "# Test" > README.md
64+
git add . && git commit -m "init"
65+
cd ..
66+
67+
# Run deepwork install to set up the project (this also runs sync)
68+
uv run deepwork install --platform claude --path test_project
69+
70+
# Validate generated commands exist
71+
echo "Checking generated commands..."
72+
ls -la test_project/.claude/commands/
73+
74+
# Verify command files exist
75+
test -f test_project/.claude/commands/fruits.identify.md || (echo "Missing fruits.identify.md" && exit 1)
76+
test -f test_project/.claude/commands/fruits.classify.md || (echo "Missing fruits.classify.md" && exit 1)
77+
78+
# Verify command content
79+
grep -q "# fruits.identify" test_project/.claude/commands/fruits.identify.md
80+
grep -q "raw_items" test_project/.claude/commands/fruits.identify.md
81+
grep -q "identified_fruits.md" test_project/.claude/commands/fruits.identify.md
82+
83+
grep -q "# fruits.classify" test_project/.claude/commands/fruits.classify.md
84+
grep -q "identified_fruits.md" test_project/.claude/commands/fruits.classify.md
85+
grep -q "classified_fruits.md" test_project/.claude/commands/fruits.classify.md
86+
87+
echo "Command generation validated successfully!"
88+
89+
# Job 2: End-to-end test with Claude Code (only when API key is available)
90+
claude-code-e2e:
91+
runs-on: ubuntu-latest
92+
needs: validate-generation
93+
if: github.event_name == 'workflow_dispatch' || github.event_name == 'schedule'
94+
env:
95+
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
96+
steps:
97+
- uses: actions/checkout@v4
98+
99+
- name: Check for API key
100+
id: check-key
101+
run: |
102+
if [ -z "$ANTHROPIC_API_KEY" ]; then
103+
echo "has_key=false" >> $GITHUB_OUTPUT
104+
echo "::warning::ANTHROPIC_API_KEY not set, skipping Claude Code e2e test"
105+
else
106+
echo "has_key=true" >> $GITHUB_OUTPUT
107+
fi
108+
109+
- name: Install Node.js (for Claude Code CLI)
110+
if: steps.check-key.outputs.has_key == 'true'
111+
uses: actions/setup-node@v4
112+
with:
113+
node-version: '20'
114+
115+
- name: Install Claude Code CLI
116+
if: steps.check-key.outputs.has_key == 'true'
117+
run: npm install -g @anthropic-ai/claude-code
118+
119+
- name: Install uv
120+
if: steps.check-key.outputs.has_key == 'true'
121+
uses: astral-sh/setup-uv@v4
122+
with:
123+
version: "latest"
124+
125+
- name: Set up Python
126+
if: steps.check-key.outputs.has_key == 'true'
127+
uses: actions/setup-python@v5
128+
with:
129+
python-version: "3.11"
130+
131+
- name: Install deepwork
132+
if: steps.check-key.outputs.has_key == 'true'
133+
run: uv sync
134+
135+
- name: Set up test project
136+
if: steps.check-key.outputs.has_key == 'true'
137+
run: |
138+
mkdir -p test_project/.deepwork/jobs
139+
mkdir -p test_project/.claude # Required for platform detection
140+
cp -r tests/fixtures/jobs/fruits test_project/.deepwork/jobs/
141+
142+
cd test_project
143+
git init
144+
git config user.email "test@test.com"
145+
git config user.name "Test"
146+
echo "# CI Test Project" > README.md
147+
git add . && git commit -m "init"
148+
cd ..
149+
150+
# Run deepwork install to set up the project (this also runs sync)
151+
uv run deepwork install --platform claude --path test_project
152+
153+
echo "Test project setup complete"
154+
ls -la test_project/.claude/commands/
155+
156+
- name: Run Claude Code - Identify Step
157+
if: steps.check-key.outputs.has_key == 'true'
158+
working-directory: test_project
159+
timeout-minutes: 5
160+
run: |
161+
# Run the identify step with a deterministic input
162+
# Using --print to output result, --yes to auto-accept
163+
claude --yes --print "/fruits.identify" <<EOF
164+
raw_items: apple, car, banana, chair, orange, table, mango, laptop
165+
EOF
166+
167+
# Verify output was created
168+
if [ -f "identified_fruits.md" ]; then
169+
echo "Identify step completed successfully!"
170+
echo "--- Output ---"
171+
cat identified_fruits.md
172+
else
173+
echo "ERROR: identified_fruits.md was not created"
174+
exit 1
175+
fi
176+
177+
- name: Run Claude Code - Classify Step
178+
if: steps.check-key.outputs.has_key == 'true'
179+
working-directory: test_project
180+
timeout-minutes: 5
181+
run: |
182+
# Run the classify step
183+
claude --yes --print "/fruits.classify"
184+
185+
# Verify output was created
186+
if [ -f "classified_fruits.md" ]; then
187+
echo "Classify step completed successfully!"
188+
echo "--- Output ---"
189+
cat classified_fruits.md
190+
else
191+
echo "ERROR: classified_fruits.md was not created"
192+
exit 1
193+
fi
194+
195+
- name: Validate outputs
196+
if: steps.check-key.outputs.has_key == 'true'
197+
working-directory: test_project
198+
run: |
199+
echo "=== Validating outputs ==="
200+
201+
# Check identified_fruits.md contains expected fruits
202+
echo "Checking identified_fruits.md..."
203+
grep -qi "apple" identified_fruits.md || (echo "Missing: apple" && exit 1)
204+
grep -qi "banana" identified_fruits.md || (echo "Missing: banana" && exit 1)
205+
grep -qi "orange" identified_fruits.md || (echo "Missing: orange" && exit 1)
206+
grep -qi "mango" identified_fruits.md || (echo "Missing: mango" && exit 1)
207+
208+
# Check classified_fruits.md has expected structure
209+
echo "Checking classified_fruits.md..."
210+
grep -qi "citrus\|tropical\|pome" classified_fruits.md || (echo "Missing fruit categories" && exit 1)
211+
212+
echo "All validations passed!"
213+
214+
- name: Upload test artifacts
215+
if: steps.check-key.outputs.has_key == 'true' && always()
216+
uses: actions/upload-artifact@v4
217+
with:
218+
name: claude-code-test-outputs
219+
path: |
220+
test_project/identified_fruits.md
221+
test_project/classified_fruits.md
222+
retention-days: 7

‎CLA/version_1/CLA_SIGNATORIES.md‎

Lines changed: 20 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,20 @@
1-
# CLA Signatories
2-
3-
This file tracks individual contributors who have signed the Contributor License Agreement (CLA) for the DeepWork project.
4-
5-
## Individual Contributors
6-
7-
| GitHub Username | Date Signed | Signature Method |
8-
|-----------------|-------------|------------------|
9-
| <!-- Add your GitHub username here --> | | |
10-
11-
---
12-
13-
## How to Sign
14-
15-
When you submit your first pull request, the CLA Assistant bot will guide you through signing the CLA electronically by commenting on your PR.
16-
17-
---
18-
19-
## Corporate Contributors
20-
21-
Organizations that have signed the Corporate CLA are tracked separately. If you are contributing on behalf of your employer, please ensure your organization has signed the Corporate CLA by contacting legal@unsupervised.com.
22-
23-
---
24-
25-
For questions about the CLA, see [CLA.md](CLA.md) or contact legal@unsupervised.com.
1+
{
2+
"signedContributors": [
3+
{
4+
"name": "nhorton",
5+
"id": 204146,
6+
"comment_id": 3752380523,
7+
"created_at": "2026-01-15T00:57:16Z",
8+
"repoId": 1132406094,
9+
"pullRequestNo": 27
10+
},
11+
{
12+
"name": "tylerwillis",
13+
"id": 50716,
14+
"comment_id": 3753520846,
15+
"created_at": "2026-01-15T08:27:44Z",
16+
"repoId": 1132406094,
17+
"pullRequestNo": 31
18+
}
19+
]
20+
}

‎tests/e2e/__init__.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
"""End-to-end tests for DeepWork with Claude Code."""

0 commit comments

Comments
 (0)