Roadmap Item 13: Add Reproducible Rule Validation Harness
Priority: P2
Area: Detection engineering
References: src/threatxtension/config/custom_semgrep_rules.yaml,
scripts/analyze_extension_patterns.py
Custom Semgrep rules are central to the project. They need a repeatable way to validate
true positives, false positives, and expected MITRE/CWE metadata.
Scope:
- Add rule fixtures with positive and negative JavaScript examples.
- Add tests that run Semgrep against fixture files.
- Validate rule metadata, severity, CWE, MITRE, and category fields.
- Document how to add a new rule safely.
Acceptance criteria:
- CI fails when custom rule syntax breaks or a fixture no longer triggers.
- Each high-value rule has at least one positive and one negative fixture.
- Rule authoring docs exist under
docs/.
Created from docs/roadmap.md.
Roadmap Item 13: Add Reproducible Rule Validation Harness
Priority: P2
Area: Detection engineering
References:
src/threatxtension/config/custom_semgrep_rules.yaml,scripts/analyze_extension_patterns.pyCustom Semgrep rules are central to the project. They need a repeatable way to validate
true positives, false positives, and expected MITRE/CWE metadata.
Scope:
Acceptance criteria:
docs/.Created from
docs/roadmap.md.