Context
Issue #47: "We should have 100% tests coverage, which should copy most of test
cases from our competitors in each sector/scope" and "We should check that
nothing is deferred or left unimplemented in our vision and roadmap." Today 48
provenanced PARITY_FIXTURES exist (a small fraction of upstream suites), the
cargo llvm-cov job enforces no threshold, and docs/parity-roadmap.md still
defers SQL dialect and Delphi-specific coverage. See
requirements.md R-19/R-20 and
solution-plans.md S-15.
Blocked by: #02, #08, #12, #14 - this is the closing gate over the
features those issues add.
Scope
- Port the five highest-value suites identified in
competitors-code-tooling.md:
Coccinelle tests/ (.c/.cocci/.res transform triples), tree-sitter
test/corpus + core test/fixtures/error_corpus, Semgrep
tests/patterns/<lang>/, srcML test/parser/testsuite lossless round-trip
cases, LibCST adversarial whitespace fixtures (runner-up: babel-parser
fixtures). Sample per construct where suites are huge; document sampling so
nothing reads as "covered everything" when it is not.
- Extend
PARITY_TARGETS with the surveyed projects not yet tracked:
ast-grep, Semgrep, Comby, GritQL, srcML, difftastic, Babel, SWC,
OpenRewrite, Spoon, JavaParser, Rascal, Stratego/Spoofax, TXL, MPS,
Coccinelle; GF, Universal Dependencies, LanguageTool from the
natural-language survey; doublets-rs and links-notation storage gates.
- Ratchet coverage: record the current
cargo llvm-cov line coverage, fail
CI below the floor, raise the floor with each wave toward 100%.
- Roadmap audit (R-19): every deferral in
docs/parity-roadmap.md is either
implemented or tracked by an open issue with a link.
Acceptance criteria
References
Filed from docs/case-studies/issue-47/proposed-issues/15-competitor-corpora-and-coverage-gate.md. Part of the implementation plan for #47.
Context
Issue #47: "We should have 100% tests coverage, which should copy most of test
cases from our competitors in each sector/scope" and "We should check that
nothing is deferred or left unimplemented in our vision and roadmap." Today 48
provenanced
PARITY_FIXTURESexist (a small fraction of upstream suites), thecargo llvm-covjob enforces no threshold, anddocs/parity-roadmap.mdstilldefers SQL dialect and Delphi-specific coverage. See
requirements.mdR-19/R-20 andsolution-plans.mdS-15.Blocked by:
#02,#08,#12,#14- this is the closing gate over thefeatures those issues add.
Scope
competitors-code-tooling.md:Coccinelle
tests/(.c/.cocci/.restransform triples), tree-sittertest/corpus+ coretest/fixtures/error_corpus, Semgreptests/patterns/<lang>/, srcMLtest/parser/testsuitelossless round-tripcases, LibCST adversarial whitespace fixtures (runner-up: babel-parser
fixtures). Sample per construct where suites are huge; document sampling so
nothing reads as "covered everything" when it is not.
PARITY_TARGETSwith the surveyed projects not yet tracked:ast-grep, Semgrep, Comby, GritQL, srcML, difftastic, Babel, SWC,
OpenRewrite, Spoon, JavaParser, Rascal, Stratego/Spoofax, TXL, MPS,
Coccinelle; GF, Universal Dependencies, LanguageTool from the
natural-language survey; doublets-rs and links-notation storage gates.
cargo llvm-covline coverage, failCI below the floor, raise the floor with each wave toward 100%.
docs/parity-roadmap.mdis eitherimplemented or tracked by an open issue with a link.
Acceptance criteria
capability gates, like existing targets.
bump: minor).References
requirements.mdR-19, R-20solution-plans.mdS-15issue-47-76af108c0f24(PR Finish issue #47 parity feature set #48).Filed from
docs/case-studies/issue-47/proposed-issues/15-competitor-corpora-and-coverage-gate.md. Part of the implementation plan for #47.