perf: reduce test execution time during task batches

[HenryLach]

Problem

Tasks spend a disproportionate amount of time running tests. The full test suite (~2500+ tests, ~150-170s) runs multiple times per task — after each implementation step and again in the final Testing & Verification step. For a 4-step task, that's potentially 5+ full suite runs (~12-15 minutes of pure test execution).

Current Behavior

• Worker runs cd extensions && npx vitest run (full suite) whenever it verifies work
• Every task's Testing step runs the full suite regardless of which files were changed
• Tests run sequentially
• No caching or change detection

Suggestions (in priority order)

1. Selective test execution (highest impact, lowest effort)

Run only tests related to changed files during implementation steps. Full suite only in the final Testing step.

Vitest supports this natively:

• npx vitest run --changed — runs tests affected by uncommitted changes
• npx vitest run tests/specific-file.test.ts — explicit file targeting

Implementation: Update the worker template and/or PROMPT conventions to:

• During implementation steps: run tests matching the files being changed
• Final Testing step: run the full suite once as a gate

The create-taskplane-task skill could generate targeted test commands in each step's artifacts section, and reserve npx vitest run (full suite) for the final Testing & Verification step only.

2. Parallel test execution (high impact, medium effort)

Vitest supports parallel execution via --pool=threads or --pool=forks. Many of our tests are source-based (pure string analysis) and would parallelize trivially. Tests that use the filesystem (temp directories, git operations) may need isolation flags.

Implementation:

• Audit test files for parallelism safety
• Add --pool=forks to the default test command
• Mark tests needing isolation with describe.sequential() or Vitest's test.sequential

3. Test caching via change detection (high impact, low effort)

Vitest's --changed flag uses git to determine which files changed since the last commit. Since workers commit at step boundaries, the changed-file set between steps is small. Only tests affected by those changes would run.

Implementation: Change the default test command in task-runner.yaml template from npx vitest run to npx vitest run --changed for intermediate checks, keeping the full suite for the final step.

4. Split test steps in PROMPT template (medium impact, low effort)

Instead of one big Testing & Verification step, have the worker run relevant tests after each implementation step (fast feedback) and only run the full suite in the final step.

Implementation: Update the create-taskplane-task skill's prompt template to include per-step test suggestions:

### Step 2: Implement feature X
- [ ] Implement the feature
- [ ] Run relevant tests: \
�[1m�[46m RUN �[49m�[22m �[36mv4.1.0 �[39m�[90mC:/dev/taskplane�[39m

### Step N-1: Testing & Verification
- [ ] Run FULL test suite: \
�[1m�[46m RUN �[49m�[22m �[36mv4.1.0 �[39m�[90mC:/dev/taskplane�[39m

5. Test categorization with tags (medium impact, higher effort)

Tag tests as fast (source-based, <1s) vs slow (filesystem, spawning, >5s). Workers run fast tests during implementation, full suite at the end.

Implementation:

• Add Vitest --include patterns or custom test naming conventions
• Source-based tests (string matching): *.source.test.ts or tagged @fast
• Integration tests (filesystem, git, tmux): *.integration.test.ts or tagged @slow
• Worker runs npx vitest run --include='**/fast*' during steps, full suite at gate

Expected Impact

Approach	Test time reduction	Effort
Selective execution	~60-70% per step	Low
Parallel execution	~40-50% overall	Medium
Change detection	~50-60% per step	Low
Split test steps	~30% (fewer full runs)	Low
Test categorization	~60-70% per step	Higher

Approaches 1-3 are complementary and could be combined for maximum effect. Recommended starting point: #1 (selective execution) — update PROMPT conventions and worker template to run targeted tests per step, full suite only at the end.

Related

• perf: supervisor input lag during batch execution — event loop contention #199 — event loop contention (test execution contributes to overall batch duration)
• create-taskplane-task skill — generates PROMPT.md test steps

[HenryLach]

Shipped in v0.15.0.