feat(reports): add the detection-coverage pillar so the generated CISO report matches the rollup skill - #102
Conversation
…on-engineer report pillars The CISO risk-rollup skill gathers a headline detection-coverage number from sentinel-mcp, but scripts/report_gather.py only had six _pillar_ factories, so the generated CISO report silently omitted it. Adds _pillar_detection_coverage (CISO: summary finding only, matching every other one-headline pillar) and _sentinel_analytics_rules (detection-engineer: full result incl. per-rule findings, where the rule inventory belongs). Adds EN/ES i18n labels for the two new group keys. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XYTy7da8Z5ZHhkwcCpjojZ
docs/user-guide/workflows.md, skills/reports/generate-report/SKILL.md, and the opencode CISO persona prompt all documented the six-pillar / "Sentinel not yet wired" divergence between the agent-driven rollup skill and the generated report. Now that report_gather.py has the seventh pillar, update all three to say seven and name Sentinel explicitly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XYTy7da8Z5ZHhkwcCpjojZ
|
Claude finished @ubercylon8's task in 1m 20s —— View job Security review — PR #102
Summary: no security-significant issues found. This is a low-risk wiring PR — it adds a 7th CISO pillar and a detection-engineer group by calling an existing, already-reviewed Checked against the CLAUDE.md Critical Rules:
Minor, non-blocking nit: the PR description references a test named No inline comments — nothing rose to the level of a specific line-level finding. |
What & why
Wires
sentinel-mcpinto the automated report generator, closing a divergenceopened when server #9 landed:
skills/cross-platform/ciso-risk-rollup/SKILL.mdgathers a detection-coverage pillar, but
scripts/report_gather.pyhad onlysix
_pillar_*factories — so the agent-driven rollup covered seven pillars whilethe generated CISO report covered six.
docs/user-guide/workflows.mddocumentedthat gap rather than hiding it; this PR removes the gap and the caveat.
CISO — seventh pillar
detection_coverage, from Sentinel'sget_detection_coverage. It answers aquestion no other pillar does: how much of ATT&CK do our own analytics rules
actually cover? — and it separates operator-authored rules (
Scheduled,NRT)from Microsoft-managed ones (
Fusion,MicrosoftSecurityIncidentCreation,MLBehaviorAnalytics,ThreatIntelligence), because the overall number flatters.On the validation workspace the honest reading was "5 rules (5 enabled, 4 custom)
— 2 of 14 MITRE tactics covered by custom rules (12 covered overall, incl.
Microsoft-managed rules)".
detection_engineer — analytics-rule inventory
That persona gathered alerts, incidents, LimaCharlie D&R rules, endpoint detections
and PA weak techniques, but had no Sentinel analytics-rule coverage at all —
a real gap for the persona whose entire job this is. Added as
analytics_rules.The design decision
get_detection_coveragereturns a summary finding plus up to 25 per-rulefindings. Every other CISO pillar contributes exactly one headline number, and 25
rule entries would swamp an executive rollup.
So the two personas consume the same tool differently:
findings[:1]), one headline like every sibling pillar.that detail belongs.
This split is mutation-tested: making the CISO factory return the full list fails
test_ciso_pillar_keeps_only_the_summary_finding.Degradation
No wrapper needed.
get_detection_coveragealready returns"Sentinel detection coverage unavailable — ARM coordinates not configured" when
the ARM triple is unset, and
sections.is_not_assessedmatches onDEGRADATION_MARKERS, which contains"not configured"— so a logs-onlydeployment renders the pillar as NOT ASSESSED and the rollup carries on, which
is exactly the "handle a dark pillar gracefully" behaviour the skill specifies.
Verified rather than assumed.
i18n
New group keys need labels in both blocks, and the EN block carries an explicit
constraint that existing English values must stay byte-identical so the frozen
goldens do not move:
group_detection_coveragegroup_analytics_rulesNo golden file changed.
Docs
docs/user-guide/workflows.md,skills/reports/generate-report/SKILL.md,.opencode/agents/ciso.md, andreport_gather.py's own module docstring andGATHER_MAPcomment all said "six". Updated. The remaining "six-pillar" referenceslive in
docs/superpowers/specs and plans, which are historical records of thedesign at the time and are deliberately untouched.
Note
ciso-risk-rollup/SKILL.md's "a partial rollup across the remaining sixpillars is still valuable" is correct as written — with seven pillars, one dark
leaves six.
Checklist (mirrors the Critical Rules in CLAUDE.md)
redact_finding.core/—scripts/report_gather.pyis the designatedplatform-aware layer (
core/reportsstays platform-free); no safety logic added.uv run pytest(1010 passed, 1 skipped),ruff,mypyclean;gen_docs.pyno diff.🤖 Generated with Claude Code
https://claude.ai/code/session_01XYTy7da8Z5ZHhkwcCpjojZ