Skip to content

Expand omv-audit and omv-repro eval coverage - #6

Merged
bx33661 merged 1 commit into
mainfrom
feat/expand-audit-repro-evals
May 10, 2026
Merged

Expand omv-audit and omv-repro eval coverage#6
bx33661 merged 1 commit into
mainfrom
feat/expand-audit-repro-evals

Conversation

@bx33661

@bx33661 bx33661 commented May 10, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds 3 new eval scenarios to omv-audit (guard-bypass, inaccessible-source, critical-ssti)
  • Adds 3 new eval scenarios to omv-repro (artifact-recording, version-mismatch, submission-ready)
  • Both skills now have 6 evals each with golden outputs and heuristic assertions

Test plan

  • All 6 new golden outputs pass check_output.py assertions
  • All 6 existing evals still pass (no regression)
  • python3 scripts/validate_skill.py — all 9 skills OK

omv-audit adds: guard-bypass (bypassable guard documentation),
inaccessible-source (blocked when repo is private), critical-ssti
(scope-changed CVSS 10.0 with S:C vector).

omv-repro adds: artifact-recording (repro artifacts with file paths),
version-mismatch (warns on tested vs installed version divergence),
submission-ready (validates all fields and reports readiness score).
@bx33661
bx33661 merged commit c29dd1b into main May 10, 2026
0 of 2 checks passed
@bx33661
bx33661 deleted the feat/expand-audit-repro-evals branch July 26, 2026 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant