Skip to content

Fix fixture matrix derived coverage - #834

Merged
chubes4 merged 2 commits into
mainfrom
fix/833-derived-fixture-coverage
Aug 4, 2026
Merged

Fix fixture matrix derived coverage#834
chubes4 merged 2 commits into
mainfrom
fix/833-derived-fixture-coverage

Conversation

@chubes4

@chubes4 chubes4 commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • replace the fixed 72-fixture policy with one derived active/solved eligibility inventory shared by discovery, preflight, and bench validation
  • enforce globally unique stable IDs across active and solved corpora before lane selection; collisions fail closed with corpus, root, and fixture-path attribution
  • preserve documented planning_empty behavior for missing, non-directory, and top-level symlink roots while retaining strict validation for real corpus roots
  • preserve target-plus-solved and solved-only semantics while preventing duplicate bare IDs from reaching artifact/result paths
  • retain coverage evidence in summary.json, cli-run.json, and additive fixture-coverage.json

Evidence

  • node --test --test-name-pattern="missing and top-level symlink roots|fixture coverage|active and solved corpora|committed SSI fixture corpus|result evidence writes|builds one-command Blocks Engine|solved-only selects exactly" tools/fixture-matrix.test.mjs (8 passed)
  • node --check lib/fixture-matrix/fixtures.mjs && node --check lib/fixture-matrix/result.mjs && node --check tools/run-fixture-matrix.mjs && node --check bench/static-site-fixture-matrix.bench.mjs && node --check tools/fixture-matrix.test.mjs
  • git diff --check
  • Focused coverage verifies missing and symlink planning-empty roots, addition, deletion/omission, malformed metadata, local and active-vs-solved duplicate IDs, solved-only selection, promotion selection, collision-safe artifact construction, legacy finding-packet consumer compatibility, and the deterministic committed SSI fixture inventory. No fixture matrix execution was run.

Compatibility

  • Valid manifest-less fixtures remain discoverable with their existing unknown taxonomy warning.
  • finding-packets.json remains the legacy top-level array; no consumer migration is required.
  • Coverage is additive in fixture-coverage.json and existing summary/CLI fields.
  • Target-plus-solved and solved-only lane selection remain unchanged for globally unique fixture IDs.
  • Missing, non-directory, and top-level symlink roots produce the documented planning-empty result instead of throwing during discovery.

Fixes #833

AI Assistance

OpenAI GPT-5.6 Sol via OpenCode implemented the derived coverage contract, review-blocker fixes, docs, and focused tests. Chris Huber reviewed and owns every line.

chubes4 added 2 commits August 4, 2026 11:31
AI assistance: OpenAI GPT-5.6 Sol via OpenCode implemented the fixture coverage contract and focused tests. Chris Huber reviewed and owns this change.
AI assistance: OpenAI GPT-5.6 Sol via OpenCode enforced global fixture IDs, restored finding packet compatibility, and added focused tests. Chris Huber reviewed and owns this change.
@chubes4
chubes4 merged commit 7762347 into main Aug 4, 2026
2 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Derive fixture-matrix corpus coverage instead of hard-coding 72 fixtures

1 participant