Skip to content

Commit 84be2c8

Browse files
fix(6d83-b949): reject low scores on dimensions with no findings in review validator
The review validator allowed dimensions with zero findings to have arbitrarily low scores (e.g., correctness=2 with no correctness findings). Per scoring rules, "no findings → score 4-5". Added validation that dimensions with no findings must score 4 or higher. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 942aacc commit 84be2c8

File tree

2 files changed

+33
-2
lines changed

2 files changed

+33
-2
lines changed

plugins/dso/scripts/validate-review-output.sh

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -289,6 +289,14 @@ if isinstance(findings, list) and isinstance(scores, dict):
289289
f"this dimension are severity='minor', which requires score 4-5 "
290290
f"(score 3 is reserved for important findings)"
291291
)
292+
# No-findings low-score check (6d83-b949): if a dimension has no findings
293+
# at all, its score must be 4 or 5 per scoring rules ("no findings → score 4-5").
294+
for dim, score in scores.items():
295+
if dim not in dim_severities and isinstance(score, (int, float)) and score < 4:
296+
errors.append(
297+
f"score '{dim}'={score} violates scoring rules: this dimension has "
298+
f"no findings, which requires score 4-5"
299+
)
292300
293301
# Validate summary
294302
summary = data.get("summary")

tests/hooks/test-validate-review-output.sh

Lines changed: 25 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -211,7 +211,7 @@ VALID_CRD_FILE=$(write_fixture "valid-crd.json" '{
211211
"design": 4,
212212
"maintainability": 4,
213213
"correctness": 4,
214-
"verification": 3
214+
"verification": 4
215215
},
216216
"findings": [],
217217
"summary": "Code is well-structured and tests are adequate."
@@ -692,7 +692,7 @@ assert_ne \
692692
NEW_DIM_VALID_FILE=$(write_fixture "new-dim-valid.json" '{
693693
"scores": {
694694
"correctness": 4,
695-
"verification": 3,
695+
"verification": 4,
696696
"hygiene": 4,
697697
"design": 4,
698698
"maintainability": 5
@@ -948,4 +948,27 @@ assert_contains \
948948
"SCHEMA_VALID: no" \
949949
"$INVALID_RISK_CATEGORY_OUTPUT"
950950

951+
# =============================================================================
952+
# Test: no-findings dimension with low score should be rejected (6d83-b949)
953+
# Scoring rules: "Minor only or no findings → score 4-5"
954+
# A dimension with zero findings and score < 4 is a scoring error.
955+
# =============================================================================
956+
echo ""
957+
echo "--- test_no_findings_dimension_low_score_rejected ---"
958+
_snapshot_fail
959+
960+
_NO_FINDINGS_LOW_SCORE_FILE=$(write_fixture "no_findings_low_score.json" '{
961+
"scores": {"hygiene": 5, "design": 5, "maintainability": 5, "correctness": 2, "verification": 4},
962+
"findings": [
963+
{"severity": "minor", "category": "verification", "description": "Minor test coverage gap", "file": "a.sh"}
964+
],
965+
"summary": "All findings are minor but correctness scored low with no correctness findings."
966+
}')
967+
_NO_FINDINGS_LOW_SCORE_EXIT=$(run_script code-review-dispatch "$_NO_FINDINGS_LOW_SCORE_FILE")
968+
assert_ne "test_no_findings_dimension_low_score_rejected" "0" "$_NO_FINDINGS_LOW_SCORE_EXIT"
969+
_NO_FINDINGS_LOW_SCORE_OUTPUT=$(bash "$SCRIPT" code-review-dispatch "$_NO_FINDINGS_LOW_SCORE_FILE" 2>&1 || true)
970+
assert_contains "test_no_findings_dimension_low_score_error_message" "no findings" "$_NO_FINDINGS_LOW_SCORE_OUTPUT"
971+
972+
assert_pass_if_clean "test_no_findings_dimension_low_score_rejected"
973+
951974
print_summary

0 commit comments

Comments
 (0)