Skip to content

Commit 6ff92e5

Browse files
spboyerCopilot
andauthored
feat: sensei scoring parity — WHEN: triggers, spec-security, Invalid level, advisory checks 16-18 (#79)
* feat: sensei scoring parity — WHEN: triggers, spec-security, Invalid level, advisory checks 16-18, context-dependent anti-triggers Implements seven features to align waza scoring with spboyer/sensei: - #72: Add 'WHEN:' trigger pattern recognition (cross-model optimized) - #73: Add spec-security checker (flags XML angle brackets and reserved prefixes) - #74: Add Invalid adherence level for descriptions >1024 chars (hard-fail) - #75: Add cross-model density checker (advisory 16: word count >60, action verb check) - #76: Add body structure quality checker (advisory 17: examples, code blocks, error handling) - #77: Add progressive disclosure checker (advisory 18: flags >500 line bodies, >50 line code blocks) - #78: Add context-dependent anti-trigger risk (Low: 1-5 skills, Moderate: 6-14, High: 15+) All changes include comprehensive test coverage. Closes #72, closes #73, closes #74, closes #75, closes #76, closes #77, closes #78 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address PR #79 review feedback — punctuation stripping, fence parsing, slice traversal, WHEN: count, body field, summary format Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address PR #79 review feedback — section-bounded counting, severity scaling, security checker nil guard, body line off-by-one, error pattern narrowing - countPhrasesAfterPattern now stops at all trigger headers (USE FOR:, WHEN:, TRIGGERS:, NOT FOR:) to prevent double-counting across sections - Anti-trigger risk severity scales with context: High→error, Moderate→warning - SpecSecurityChecker checks reserved name prefixes even when FrontmatterRaw is nil (programmatically constructed skills) - ProgressiveDisclosureChecker trims trailing newline before line count to avoid off-by-one at the 500-line boundary - BodyStructureChecker error handling patterns narrowed from broad "error" substring to heading-aware patterns ("## error", "error handling", "## troubleshooting") to reduce false positives - Updated display and loop golden strings for new spec-security and advisory 16-18 checkers - All 40 test packages pass Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: hoist regex to package level, pre-compute ToUpper, fix test label - Move numberedStepsPattern regex to package-level var (was recompiled per Check() call) - Pre-compute strings.ToUpper(after) in countPhrasesAfterPattern to avoid redundant uppercasing on each loop iteration - Fix boundary test name: 48 lines, not 50 Addresses wbreza review feedback on PR #79. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor: Rename actionVerbs to commonLeadWords in advisory checks * fix: address review feedback and golden string mismatches on PR #79 - Update golden strings: 'no action verbs' → 'no common lead words' - Clarify spec-security summary when FrontmatterRaw is nil - Add TODO for plumbing SkillCount into production callers Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 51d6638 commit 6ff92e5

9 files changed

Lines changed: 998 additions & 29 deletions

File tree

cmd/waza/dev/display_test.go

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ Issues:
3838
✅ spec-name: Name follows spec naming rules
3939
✅ spec-dir-match: Cannot validate (missing path or name)
4040
✅ spec-description: Description is valid
41+
✅ spec-security: No security risks detected in frontmatter
4142
✅ spec-compatibility: No compatibility field (optional)
4243
⚠️ spec-license: No license field found
4344
📎 Best practice: include a license field (e.g., MIT, Apache-2.0)
@@ -48,8 +49,11 @@ Issues:
4849
✅ module-count: Found 0 reference module(s)
4950
✅ complexity: Complexity: compact (14 tokens, 0 modules)
5051
✅ negative-delta-risk: No negative delta risk patterns detected
51-
⚠️ procedural-content: Description lacks procedural language (no action verbs or procedure keywords found)
52+
⚠️ procedural-content: Description lacks procedural language (no common lead words or procedure keywords found)
5253
✅ over-specificity: No over-specificity patterns detected
54+
✅ cross-model-density: Advisory 16: first sentence doesn't lead with action verb (reduces clarity)
55+
⚠️ body-structure: Advisory 17: body structure quality — body lacks actionable instructions (no code blocks, numbered steps, or commands); no examples section found; no error handling or troubleshooting section found
56+
✅ progressive-disclosure: Content structure supports progressive disclosure
5357
`
5458
require.Equal(t, want, buf.String())
5559
}
@@ -79,6 +83,7 @@ Anti-triggers: 1
7983
✅ spec-name: Name follows spec naming rules
8084
✅ spec-dir-match: Cannot validate (missing path or name)
8185
✅ spec-description: Description is valid
86+
✅ spec-security: No security risks detected in frontmatter
8287
✅ spec-compatibility: No compatibility field (optional)
8388
⚠️ spec-license: No license field found
8489
📎 Best practice: include a license field (e.g., MIT, Apache-2.0)
@@ -96,6 +101,9 @@ MCP Integration: 1/4
96101
✅ negative-delta-risk: No negative delta risk patterns detected
97102
✅ procedural-content: Description contains procedural language
98103
✅ over-specificity: No over-specificity patterns detected
104+
✅ cross-model-density: Advisory 16: first sentence doesn't lead with action verb (reduces clarity)
105+
⚠️ body-structure: Advisory 17: body structure quality — body lacks actionable instructions (no code blocks, numbered steps, or commands); no examples section found; no error handling or troubleshooting section found
106+
✅ progressive-disclosure: Content structure supports progressive disclosure
99107
`
100108
require.Equal(t, want, buf.String())
101109
}
@@ -123,6 +131,7 @@ Anti-triggers: 1
123131
✅ spec-name: Name follows spec naming rules
124132
✅ spec-dir-match: Cannot validate (missing path or name)
125133
✅ spec-description: Description is valid
134+
✅ spec-security: No security risks detected in frontmatter
126135
✅ spec-compatibility: No compatibility field (optional)
127136
⚠️ spec-license: No license field found
128137
📎 Best practice: include a license field (e.g., MIT, Apache-2.0)
@@ -135,6 +144,9 @@ Anti-triggers: 1
135144
✅ negative-delta-risk: No negative delta risk patterns detected
136145
✅ procedural-content: Description contains procedural language
137146
✅ over-specificity: No over-specificity patterns detected
147+
🌟 cross-model-density: Description density is optimal for cross-model use
148+
⚠️ body-structure: Advisory 17: body structure quality — body lacks actionable instructions (no code blocks, numbered steps, or commands); no examples section found; no error handling or troubleshooting section found
149+
✅ progressive-disclosure: Content structure supports progressive disclosure
138150
`
139151
require.Equal(t, want, buf.String())
140152
}

cmd/waza/dev/loop_test.go

Lines changed: 35 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,7 @@ Anti-triggers: 0
109109
✅ spec-name: Name follows spec naming rules
110110
✅ spec-dir-match: Directory name matches skill name
111111
✅ spec-description: Description is valid
112+
✅ spec-security: No security risks detected in frontmatter
112113
✅ spec-compatibility: No compatibility field (optional)
113114
⚠️ spec-license: No license field found
114115
📎 Best practice: include a license field (e.g., MIT, Apache-2.0)
@@ -121,6 +122,9 @@ Anti-triggers: 0
121122
✅ negative-delta-risk: No negative delta risk patterns detected
122123
✅ procedural-content: Description contains procedural language
123124
✅ over-specificity: No over-specificity patterns detected
125+
🌟 cross-model-density: Description density is optimal for cross-model use
126+
⚠️ body-structure: Advisory 17: body structure quality — body lacks actionable instructions (no code blocks, numbered steps, or commands); no examples section found; no error handling or troubleshooting section found
127+
✅ progressive-disclosure: Content structure supports progressive disclosure
124128
`
125129
require.Equal(t, expected, buf.String())
126130
}
@@ -208,6 +212,7 @@ Anti-triggers: 1
208212
✅ spec-name: Name follows spec naming rules
209213
✅ spec-dir-match: Directory name matches skill name
210214
✅ spec-description: Description is valid
215+
✅ spec-security: No security risks detected in frontmatter
211216
✅ spec-compatibility: No compatibility field (optional)
212217
⚠️ spec-license: No license field found
213218
📎 Best practice: include a license field (e.g., MIT, Apache-2.0)
@@ -225,6 +230,9 @@ MCP Integration: 1/4
225230
✅ negative-delta-risk: No negative delta risk patterns detected
226231
✅ procedural-content: Description contains procedural language
227232
✅ over-specificity: No over-specificity patterns detected
233+
✅ cross-model-density: Advisory 16: first sentence doesn't lead with action verb (reduces clarity)
234+
⚠️ body-structure: Advisory 17: body structure quality — body lacks actionable instructions (no code blocks, numbered steps, or commands); no examples section found; no error handling or troubleshooting section found
235+
✅ progressive-disclosure: Content structure supports progressive disclosure
228236
229237
✅ Target adherence level High reached!
230238
`
@@ -278,6 +286,7 @@ Issues:
278286
✅ spec-name: Name follows spec naming rules
279287
✅ spec-dir-match: Directory name matches skill name
280288
✅ spec-description: Description is valid
289+
✅ spec-security: No security risks detected in frontmatter
281290
✅ spec-compatibility: No compatibility field (optional)
282291
⚠️ spec-license: No license field found
283292
📎 Best practice: include a license field (e.g., MIT, Apache-2.0)
@@ -288,8 +297,11 @@ Issues:
288297
✅ module-count: Found 0 reference module(s)
289298
✅ complexity: Complexity: compact (15 tokens, 0 modules)
290299
✅ negative-delta-risk: No negative delta risk patterns detected
291-
⚠️ procedural-content: Description lacks procedural language (no action verbs or procedure keywords found)
300+
⚠️ procedural-content: Description lacks procedural language (no common lead words or procedure keywords found)
292301
✅ over-specificity: No over-specificity patterns detected
302+
✅ cross-model-density: Advisory 16: first sentence doesn't lead with action verb (reduces clarity)
303+
⚠️ body-structure: Advisory 17: body structure quality — body lacks actionable instructions (no code blocks, numbered steps, or commands); no examples section found; no error handling or troubleshooting section found
304+
✅ progressive-disclosure: Content structure supports progressive disclosure
293305
294306
📝 Suggested improvement (description-length):
295307
────────────────────────────────────────
@@ -385,6 +397,7 @@ Issues:
385397
✅ spec-name: Name follows spec naming rules
386398
✅ spec-dir-match: Directory name matches skill name
387399
✅ spec-description: Description is valid
400+
✅ spec-security: No security risks detected in frontmatter
388401
✅ spec-compatibility: No compatibility field (optional)
389402
⚠️ spec-license: No license field found
390403
📎 Best practice: include a license field (e.g., MIT, Apache-2.0)
@@ -395,8 +408,11 @@ Issues:
395408
✅ module-count: Found 0 reference module(s)
396409
✅ complexity: Complexity: compact (31 tokens, 0 modules)
397410
✅ negative-delta-risk: No negative delta risk patterns detected
398-
⚠️ procedural-content: Description lacks procedural language (no action verbs or procedure keywords found)
411+
⚠️ procedural-content: Description lacks procedural language (no common lead words or procedure keywords found)
399412
✅ over-specificity: No over-specificity patterns detected
413+
✅ cross-model-density: Advisory 16: first sentence doesn't lead with action verb (reduces clarity)
414+
⚠️ body-structure: Advisory 17: body structure quality — body lacks actionable instructions (no code blocks, numbered steps, or commands); no examples section found; no error handling or troubleshooting section found
415+
✅ progressive-disclosure: Content structure supports progressive disclosure
400416
401417
📝 Suggested improvement (description-length):
402418
────────────────────────────────────────
@@ -420,6 +436,7 @@ Anti-triggers: 0
420436
✅ spec-name: Name follows spec naming rules
421437
✅ spec-dir-match: Directory name matches skill name
422438
✅ spec-description: Description is valid
439+
✅ spec-security: No security risks detected in frontmatter
423440
✅ spec-compatibility: No compatibility field (optional)
424441
⚠️ spec-license: No license field found
425442
📎 Best practice: include a license field (e.g., MIT, Apache-2.0)
@@ -432,6 +449,9 @@ Anti-triggers: 0
432449
✅ negative-delta-risk: No negative delta risk patterns detected
433450
✅ procedural-content: Description contains procedural language
434451
✅ over-specificity: No over-specificity patterns detected
452+
✅ cross-model-density: Advisory 16: first sentence doesn't lead with action verb (reduces clarity)
453+
⚠️ body-structure: Advisory 17: body structure quality — body lacks actionable instructions (no code blocks, numbered steps, or commands); no examples section found; no error handling or troubleshooting section found
454+
✅ progressive-disclosure: Content structure supports progressive disclosure
435455
436456
📝 Suggested improvement (triggers):
437457
────────────────────────────────────────
@@ -455,6 +475,7 @@ Anti-triggers: 0
455475
✅ spec-name: Name follows spec naming rules
456476
✅ spec-dir-match: Directory name matches skill name
457477
✅ spec-description: Description is valid
478+
✅ spec-security: No security risks detected in frontmatter
458479
✅ spec-compatibility: No compatibility field (optional)
459480
⚠️ spec-license: No license field found
460481
📎 Best practice: include a license field (e.g., MIT, Apache-2.0)
@@ -467,6 +488,9 @@ Anti-triggers: 0
467488
✅ negative-delta-risk: No negative delta risk patterns detected
468489
✅ procedural-content: Description contains procedural language
469490
✅ over-specificity: No over-specificity patterns detected
491+
✅ cross-model-density: Advisory 16: first sentence doesn't lead with action verb (reduces clarity)
492+
⚠️ body-structure: Advisory 17: body structure quality — body lacks actionable instructions (no code blocks, numbered steps, or commands); no examples section found; no error handling or troubleshooting section found
493+
✅ progressive-disclosure: Content structure supports progressive disclosure
470494
471495
📝 Suggested improvement (anti-triggers):
472496
────────────────────────────────────────
@@ -490,6 +514,7 @@ Anti-triggers: 2
490514
✅ spec-name: Name follows spec naming rules
491515
✅ spec-dir-match: Directory name matches skill name
492516
✅ spec-description: Description is valid
517+
✅ spec-security: No security risks detected in frontmatter
493518
✅ spec-compatibility: No compatibility field (optional)
494519
⚠️ spec-license: No license field found
495520
📎 Best practice: include a license field (e.g., MIT, Apache-2.0)
@@ -502,6 +527,9 @@ Anti-triggers: 2
502527
✅ negative-delta-risk: No negative delta risk patterns detected
503528
✅ procedural-content: Description contains procedural language
504529
✅ over-specificity: No over-specificity patterns detected
530+
✅ cross-model-density: Advisory 16: first sentence doesn't lead with action verb (reduces clarity)
531+
⚠️ body-structure: Advisory 17: body structure quality — body lacks actionable instructions (no code blocks, numbered steps, or commands); no examples section found; no error handling or troubleshooting section found
532+
✅ progressive-disclosure: Content structure supports progressive disclosure
505533
506534
📝 Suggested improvement (routing-clarity):
507535
────────────────────────────────────────
@@ -584,6 +612,7 @@ Issues:
584612
✅ spec-name: Name follows spec naming rules
585613
✅ spec-dir-match: Directory name matches skill name
586614
✅ spec-description: Description is valid
615+
✅ spec-security: No security risks detected in frontmatter
587616
✅ spec-compatibility: No compatibility field (optional)
588617
⚠️ spec-license: No license field found
589618
📎 Best practice: include a license field (e.g., MIT, Apache-2.0)
@@ -594,8 +623,11 @@ Issues:
594623
✅ module-count: Found 0 reference module(s)
595624
✅ complexity: Complexity: compact (17 tokens, 0 modules)
596625
✅ negative-delta-risk: No negative delta risk patterns detected
597-
⚠️ procedural-content: Description lacks procedural language (no action verbs or procedure keywords found)
626+
⚠️ procedural-content: Description lacks procedural language (no common lead words or procedure keywords found)
598627
✅ over-specificity: No over-specificity patterns detected
628+
✅ cross-model-density: Advisory 16: first sentence doesn't lead with action verb (reduces clarity)
629+
⚠️ body-structure: Advisory 17: body structure quality — body lacks actionable instructions (no code blocks, numbered steps, or commands); no examples section found; no error handling or troubleshooting section found
630+
✅ progressive-disclosure: Content structure supports progressive disclosure
599631
600632
📝 Suggested improvement (description-length):
601633
────────────────────────────────────────

0 commit comments

Comments
 (0)