Summary
tobira has grown from ~20 to ~76 detection patterns over the course of development. Research into SIEM/security alert-fatigue practice (a well-quantified, directly analogous domain — alert volume growing by adding more detection rules over time) found: false-positive/alert-fatigue rates are NOT primarily caused by having more rules; they're caused by adding rules without a parallel investment in per-rule precision tracking. One cited real-world case: an org generating 847 alerts/day tuned to 43/day over three months not by deleting rules, but by adding risk-scoring/thresholding on top of the existing rule set, and saw true-positive rate rise from 2% to 31%.
This maps onto tobira directly, and is also the root cause behind the separate #291 finding (find_best()/efficiency_gaps() letting high-frequency base keys monopolize suggestions regardless of habit severity) — that issue's scoring-normalization half and this issue may be worth designing together.
Why this needs review before implementation
tobira already collects the raw data needed for this (usage.json tracks shown counts per command; adoption is inferred behaviorally via existing mastery-progression tracking). What doesn't exist is a systematic way to ask "of the ~76 patterns, which ones get shown often but adopted rarely?" and deprioritize/reweight those — the tobira equivalent of an alert-tuning pass. This is a genuine design question: what counts as "adopted" for a given pattern (matches the existing mastery-detection heuristic, or needs its own signal?), how much history is needed before a pattern's precision is trustworthy, and how a low-precision pattern should be deprioritized (lower score weight? higher shown-count suppression threshold? something else?).
Detectability note
N/A — the raw signal already exists in usage.json; this is a scoring/prioritization design question.
Relevant files
lua/tobira/core/graph.lua (find_best, efficiency_gaps)lua/tobira/core/logger.lua (usage.json schema, shown tracking)
Summary
tobira has grown from ~20 to ~76 detection patterns over the course of development. Research into SIEM/security alert-fatigue practice (a well-quantified, directly analogous domain — alert volume growing by adding more detection rules over time) found: false-positive/alert-fatigue rates are NOT primarily caused by having more rules; they're caused by adding rules without a parallel investment in per-rule precision tracking. One cited real-world case: an org generating 847 alerts/day tuned to 43/day over three months not by deleting rules, but by adding risk-scoring/thresholding on top of the existing rule set, and saw true-positive rate rise from 2% to 31%.
This maps onto tobira directly, and is also the root cause behind the separate #291 finding (
find_best()/efficiency_gaps()letting high-frequency base keys monopolize suggestions regardless of habit severity) — that issue's scoring-normalization half and this issue may be worth designing together.Why this needs review before implementation
tobira already collects the raw data needed for this (
usage.jsontracksshowncounts per command; adoption is inferred behaviorally via existing mastery-progression tracking). What doesn't exist is a systematic way to ask "of the ~76 patterns, which ones get shown often but adopted rarely?" and deprioritize/reweight those — the tobira equivalent of an alert-tuning pass. This is a genuine design question: what counts as "adopted" for a given pattern (matches the existing mastery-detection heuristic, or needs its own signal?), how much history is needed before a pattern's precision is trustworthy, and how a low-precision pattern should be deprioritized (lower score weight? highershown-count suppression threshold? something else?).Detectability note
N/A — the raw signal already exists in
usage.json; this is a scoring/prioritization design question.Relevant files
lua/tobira/core/graph.lua(find_best,efficiency_gaps)lua/tobira/core/logger.lua(usage.jsonschema,showntracking)