refactor(contracts): single source of truth for evidence/submission scoring - #19
Merged
Merged
Conversation
…s + add submission_score section Break the single ambiguous 'CVE readiness score' concept into two explicitly documented scores: - evidence_score (field completeness, 0-100, never penalized) - submission_score (evidence_score minus deductions/clamping, 0-100, the actual gate at >=75) Changes: - contracts/evidence.v1.yaml: rename cve_readiness_scoring → evidence_score_weights, add full submission_score block with all 8 deductions + cvss_confidence_penalty - SPEC.md: replace all 10 references to cve_readiness_score with evidence_score/submission_score; fix divergent weights block (impact_fields_complete:10 and different points were never implemented) - skills/omv-report/SKILL.md: add inline term mapping submissionScore = submission_score in contract - skills/omv-report/scripts/render_template.py: re-implement _compute_scores() to exactly mirror CLI submissionsDeductions logic (was off by 4 deduction values + missing 3 deductions + unused 'missing tested version' deduction)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
The project had three overlapping scoring concepts (
evidence_scorein CLI,submission_scorein CLI,cve_readiness_scoreonly in SPEC/comments), three naming conventions, and a Python renderer that silently disagreed with the TypeScript CLI on four deduction values plus three missing deductions and one spurious one. The same YAML produced different numbers depending on the path.This PR makes the CLI the single canonical source and propagates it everywhere. No CLI logic is changed — this is a cleanup of the contract, SPEC, skills, and Python renderer to mirror what the CLI already does.
Changes
contracts/evidence.v1.yamlcve_readiness_scoring:block →evidence_score_weights:(it already matchedcomputeEvidenceScoreinfindings.tsexactly).submission_score:block listing every deduction id + points + cap, pluscvss_confidence_penaltyrules, mirroringSUBMISSION_DEDUCTIONSandcvssConfidencePenaltyinfindings.ts.submission_score(the ≥75 gate) rather than the ambiguous 'CVE readiness score'.SPEC.mdcve_readiness_scorewithevidence_score/submission_scoreas appropriate (the ≥75 gate is always onsubmission_score).cve_readiness_scoringblock at line 247 with the canonical one matching the contract. Remove the never-implementedimpact_fields_complete: 10field, fixtested_version: 20(was incorrectly15in SPEC),local_reproducer: 15(was20), and renameduplicate_search_done→dedup_searched.skills/omv-report/SKILL.mdsubmissionScoreassubmission_scoreper the contract when it appears in the preflight block.skills/omv-report/scripts/render_template.py_compute_scores()to mirrorsrc/cli/findings.ts computeSubmissionScoreexactly. Fixed values:unverified_fieldstoFindingdataclass and load it fromprovenance.unverified_fieldsso the penalty has a real input.Verification
npm run build— passesnpm test— all 37 tests passpython3 skills/omv-report/scripts/render_template.py --finding .omv/findings/go-webhookx-ssrf.yaml --format vuldb --force— rendersevidence: 95/100 | submission: 50/100matching CLI math: 95 (10 evidence fields set, no vendor_contacted) − 10 (unknown affected_range) − 15 (5 unverified fields × 3, capped at 20) − 20 (confidence unknown) − 10 (confirmed-below-threshold bonus) = 50. ✓grep -rn cve_readiness_score contracts/ skills/ SPEC.md— zero hits in tracked filesFollow-up PR (Phase 2, not in this one)
Split
src/cli/findings.ts(1603 lines) intovalidate.ts+workflow.ts+threatmap.ts+repro.ts, and consolidate rendering inrender.ts. The contract now referencessrc/cli/findings.ts computeSubmissionScoredeliberately so Phase 1 is self-consistent before Phase 2 moves things around.Checklist
cve_readiness_scorereferences in tracked files