Skip to content

Commit a124f86

Browse files
committed
feat(contracts): rename cve_readiness_scoring → evidence_score_weights + add submission_score section
Break the single ambiguous 'CVE readiness score' concept into two explicitly documented scores: - evidence_score (field completeness, 0-100, never penalized) - submission_score (evidence_score minus deductions/clamping, 0-100, the actual gate at >=75) Changes: - contracts/evidence.v1.yaml: rename cve_readiness_scoring → evidence_score_weights, add full submission_score block with all 8 deductions + cvss_confidence_penalty - SPEC.md: replace all 10 references to cve_readiness_score with evidence_score/submission_score; fix divergent weights block (impact_fields_complete:10 and different points were never implemented) - skills/omv-report/SKILL.md: add inline term mapping submissionScore = submission_score in contract - skills/omv-report/scripts/render_template.py: re-implement _compute_scores() to exactly mirror CLI submissionsDeductions logic (was off by 4 deduction values + missing 3 deductions + unused 'missing tested version' deduction)
1 parent 51f0a92 commit a124f86

3 files changed

Lines changed: 82 additions & 31 deletions

File tree

contracts/evidence.v1.yaml

Lines changed: 30 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,11 @@
1111
# - Use "unknown" for unverified fields; never fill from memory.
1212
# - List unverified fields under provenance.unverified_fields.
1313
# - omv-report must preserve all unverified markers; never silently upgrade them.
14-
# - CVE readiness score 0-100 based on field completeness; threshold >=75 is necessary but not sufficient.
14+
# - submission_score (computed, 0-100) = evidence_score minus penalties (blockers,
15+
# missing observed_result, weak version boundaries, incomplete dedup, blocked/disproven
16+
# verdict, missing repro artifacts, unverified fields, low confidence in verdict).
17+
# Threshold: submission_score >= 75 plus validation ok and status confirmed is required
18+
# before suggesting promotion to /omv-report.
1519
# - CLI validation is the machine gate for status transitions; confirmed findings must satisfy field-level checks.
1620
# - Unknown values in important fields must be listed in provenance.unverified_fields until verified.
1721

@@ -104,26 +108,11 @@ provenance:
104108
unverified_fields: [] # list field paths that carry "unknown" or unconfirmed data; checked by CLI validation
105109
tool_versions: {} # e.g. {omv-find: "1.0"}
106110

107-
# ── Evidence and Submission Scores (computed, not stored) ─────────────────────
108-
# omv computes evidence_score from field completeness.
109-
# omv computes submission_score by penalizing unresolved blockers, unknown
110-
# observed_result, unknown affected range, incomplete dedup, and blocked/disproven
111-
# verdicts. A high evidence_score is not enough for submission-ready output.
111+
# ── Evidence Score (evidence_score) ──────────────────────────────────
112+
# evidence_score = sum of weights below; field-completeness measure, 0-100.
113+
# Never penalized, never clamped. Computed by CLI src/cli/findings.ts computeEvidenceScore.
112114
#
113-
# Evidence scoring guide (total 100):
114-
# tested version present +20
115-
# source identified +10
116-
# sink identified +10
117-
# guard missing confirmed +10
118-
# local reproducer written +15
119-
# observed result documented +10
120-
# cvss vector present +10
121-
# dedup search completed +10
122-
# vendor contacted +5
123-
#
124-
# Threshold: submission_score >= 75 plus validation OK is required before
125-
# suggesting promotion to confirmed.
126-
cve_readiness_scoring:
115+
evidence_score_weights:
127116
tested_version: 20
128117
source_identified: 10
129118
sink_identified: 10
@@ -134,4 +123,24 @@ cve_readiness_scoring:
134123
dedup_searched: 10
135124
vendor_contacted: 5
136125
total: 100
137-
threshold: 75
126+
127+
# ── Submission Score (submission_score) ────────────────────────────────
128+
# submission_score = max(0, min(100, evidence_score - Σdeductions - cvss_confidence_penalty)).
129+
# Blocked findings: submission_score = 0.
130+
# Computed by CLI src/cli/findings.ts computeSubmissionScore. Must mirror findings.ts exactly.
131+
submission_score:
132+
threshold: 75 # confirmed + validation.ok + submission_score >= 75 → report-ready
133+
cvss_confidence_penalty:
134+
per_unverified_field: 3 # capped at 20
135+
confidence_medium: 5
136+
confidence_low: 15
137+
confidence_unknown: 20
138+
deductions:
139+
missingObservedResult: 25
140+
unresolvedBlockers: 30
141+
unknownAffectedRange: 10
142+
incompleteDedup: 15
143+
blockedOrDisproven: 50
144+
plausibleExploitability: 10
145+
confirmedBelowThreshold: 10
146+
missingReproArtifacts: 10

skills/omv-report/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ Use the validation result to choose output mode:
5555
- Validation errors: lead with the errors and blockers; do not produce a submission-ready VulDB/CVE/GHSA/OSV report.
5656
- `status: blocked`: explain blockers and minimum evidence needed.
5757
- `status: candidate`: produce only triage notes or a draft outline clearly marked not ready for submission.
58-
- `status: confirmed`: proceed only if required evidence is present and submission score is at least 75/100; include validation warnings in the pre-submission checklist.
58+
- `status: confirmed`: proceed only if required evidence is present and submission score is at least 75/100 (`submissionScore` = `submission_score` in contract — the gating score after deducting blockers/unverified fields/confidence penalties); include validation warnings in the pre-submission checklist.
5959
- `submissionScore` below 75 or `verdict.exploitability` not `proven`: do not produce a submission-ready report; explain what evidence or reproduction artifact is missing.
6060
- `evidence.repro_artifacts` present: reference the artifacts as local reviewer evidence. If absent, warn that the report depends only on inline reproducer text.
6161

skills/omv-report/scripts/render_template.py

Lines changed: 51 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,7 @@ class Finding:
120120
# Provenance
121121
verification_date: str = ""
122122
researcher: str = ""
123+
unverified_fields: list[str] = field(default_factory=list)
123124
# Scores (computed)
124125
evidence_score: int = 0
125126
submission_score: int = 0
@@ -194,13 +195,22 @@ def load_finding(path: Path) -> Finding:
194195
prov = data.get("provenance") or {}
195196
f.verification_date = str(prov.get("verification_date", ""))
196197
f.researcher = str(prov.get("researcher", ""))
198+
uf = prov.get("unverified_fields")
199+
if isinstance(uf, list):
200+
f.unverified_fields = [str(x) for x in uf]
197201

198202
f.evidence_score, f.submission_score = _compute_scores(f)
199203
return f
200204

201205

202206
def _compute_scores(f: Finding) -> tuple[int, int]:
203-
"""Mirror contracts/evidence.v1.yaml scoring guide."""
207+
"""Evidence and submission scores. MUST mirror src/cli/findings.ts computeSubmissionScore exactly.
208+
209+
evidence_score = field completeness only, 0-100, never penalized or clamped.
210+
submission_score = max(0, min(100, evidence_score - deductions - cvss_penalty)).
211+
Blocked findings have submission_score = 0.
212+
"""
213+
# evidence_score — sum of known non-"unknown" fields
204214
ev = 0
205215
if is_set(f.tested):
206216
ev += 20
@@ -221,23 +231,55 @@ def _compute_scores(f: Finding) -> tuple[int, int]:
221231
if f.vendor_contacted:
222232
ev += 5
223233

234+
# submission_score — starts from evidence_score, subtracts deductions
224235
sub = ev
225-
if f.blockers:
226-
sub -= min(30, len(f.blockers) * 15)
227236
if not is_set(f.observed_result):
228-
sub -= 20
229-
if not is_set(f.affected_range):
230-
sub -= 15
237+
sub -= 25
238+
if f.blockers:
239+
sub -= 30
240+
if not is_set(f.affected_range) or f.affected_range.strip().lower() == "unknown":
241+
sub -= 10
231242
if not (f.nvd_searched and f.ghsa_searched and f.ecosystem_db_searched):
243+
sub -= 15
244+
if f.exploitability in ("blocked", "disproven"):
245+
sub -= 50
246+
if f.exploitability == "plausible":
232247
sub -= 10
248+
if f.repro_artifacts and not _any_repro_artifact_exists(f):
249+
sub -= 10
250+
251+
# cvss confidence penalty
252+
unverified_count = len(getattr(f, 'unverified_fields', []))
253+
penalty = min(20, unverified_count * 3)
254+
if f.confidence == "medium":
255+
penalty += 5
256+
elif f.confidence == "low":
257+
penalty += 15
258+
elif f.confidence == "unknown":
259+
penalty += 20
260+
sub -= penalty
261+
262+
# blocked → 0
233263
if f.exploitability in ("blocked", "disproven"):
234-
sub -= 30
235-
if not is_set(f.tested):
236-
sub -= 20
264+
sub = 0
265+
elif f.status == "blocked":
266+
sub = 0
267+
268+
# extra -10 for confirmed findings still below threshold
269+
if f.status == "confirmed" and sub < 75:
270+
sub -= 10
237271

238272
return ev, max(0, min(100, sub))
239273

240274

275+
def _any_repro_artifact_exists(f: Finding) -> bool:
276+
"""Check if any listed repro_artifact file path exists on disk (CLI's existingArtifactPaths logic)."""
277+
for ap in f.repro_artifacts:
278+
if ap and Path(ap).exists():
279+
return True
280+
return False
281+
282+
241283
def _score_line(f: Finding) -> str:
242284
return (
243285
f"Rendered by omv render_template | evidence: {f.evidence_score}/100"

0 commit comments

Comments
 (0)