EPIC: Three tiers of compounding: note, reminder, skill

[jeremymanning]

From the audit in notes/2026-09-02-audit-and-replan.md (five parallel read-only agents,
2026-09-02). Synthesis and plan, quoted from that note.

Synthesis

The system has exactly one output path, the forge, and that path costs a median 3.3 hours
and 8 agents per skill while the cheapest forms of compounding (a line in CLAUDE.md, a
reminder injected when a matching prompt or command appears) have no mechanism at all.
SKILL.md line 44 says "write a note or update CLAUDE.md" and names no path, no CLI, no
ledger row. It has been taken zero times in ten days. The insight queue has 57 candidates
in and 0 out. Two paid CANDIDATE verdicts produced nothing. The one artifact that both
accumulates and fires automatically is the repeat store, and it is the smallest, dumbest,
most deterministic piece: a content-addressed signature matched on PreToolUse. That is the
shape the rest should copy.

Second diagnosis: the measurement layer that would tell us whether any of this works is
either dead (reminder-conversion counter cannot parse its own tally), inflated
(skillreport credits the same uses to three forge rows), or never invoked (skillforge verdict, 0 rows in 807; 5 of 6 closed forges have no apply row). We cannot tune anything
until it reports.

Third: the forge protocol is 12 steps and 28 doctrine gates, and the audit could attribute
observed catches to only four cheap pieces (parse gate, routing gate, round 1 plus one
confirming round, non-fork reviewer). The orchestrator layer catches nothing and caused
the 86-hour stuck forge. Stage E has found two meta findings ever. Rounds 3..N are ~60% of
wall clock and the round record shows repairs creating new findings as often as closing
them.

Plan: three tiers of compounding, one promotion path, honest instruments

Tier 0, NOTE. A dated line in a marker block of ./.claude/CLAUDE.md,
~/.claude/CLAUDE.md, or a project memory file. Seconds. Written by skillnote (new
CLI), which also writes a note ledger row. The forge's "cheap branch" and every
CANDIDATE verdict land here.

Tier 1, REMINDER. A searchable store (<state>/reminders.jsonl) of short prose keyed
on keywords, paths, or command signatures; a UserPromptSubmit/PreToolUse hook injects
matches as additionalContext. Generalises what repeat-gate already does for command
signatures. Written by skillnote --remind, and by the insight queue when a candidate
recurs.

Tier 2, SKILL. SKILL.md plus optional scripts, forged by ONE builder and ONE cold
reviewer, default two rounds, parse gate and routing gate kept, orchestrator layer dropped
for two-round forges, judge folded into the brief, repro optional. Target: under 30
minutes for a narrow skill. Escalate rounds only on a converging record; hard cap.

Promotion: queue -> note -> reminder -> skill, driven by recurrence counted the way
the repeat store counts it. skillinsight promote moves a candidate up one tier.

Instruments: fix the unary counter, dedupe skillreport, wire or remove skillforge verdict, index the lost verdict, add skillforge doctor and a TTL reaper for stuck
forges.

Status

Waves 1 through 4 are committed: 03773ad, 7507a0b, 15b3b28, c06eb6c. The checklist
that stood here is replaced by what each issue actually reached, verified against the tree at
385624f on 2026-09-03. Implemented means the code is in and has tests. Live-proven
means it ran against a real install and produced an artifact somebody read. Measured
means a number exists that a later run can be compared against.

issue	implemented	live-proven	measured	notes
#24 measurement fixes	yes	yes, skillreport computes on 115 real counters	yes, REUSE 5 of 6 counted once per skill	closed; verdict wiring unexercised, moved to #34
#25 doctor and reaper	yes, 41 tests	yes, skillforge doctor 8 pass 0 fail on this install	n/a	closed; the reaper has never fired on a real stuck forge
#26 installer doctrine stanza	yes	yes, block at ~/.claude/CLAUDE.md:96-144, rerun reports already current	n/a	closed
#29 trivia	yes	yes, both figures derived rather than restated	n/a	closed
#30 conversion baseline	n/a	n/a	yes, 10.5% over 1456 transcripts	open; sweep is not yet a script, and the next one must count the tiers separately
#20 skillnote (T0)	yes, 73 tests	yes, 18 note rows, 4 dated lines in .claude/CLAUDE.md	yes, memory read-back 3/3 via the index, 0/3 without	closed
#21 reminder store (T1)	yes, 73 tests	yes, 16 delivery rows, both arms 3/3 on 2.1.259	yes, 49 to 66 ms per event on 500 rows	closed; prune and bounds moved to #33
#22 forge diet (T2)	yes	no	no	closed as built; no forge has run under it, see #34
#23 promotion path	yes	yes, queue drained from 57 to 0 pending	yes, 46 declined, 12 promoted	closed
#27 repeat-gate	yes, refusal arm default off	yes, 10 threshold signatures denied 0	yes, via --eligible-of	closed; reopens if a non-allowlisted signature reaches threshold
#28 doc-gate, skillcontrib, skillrepeat	yes, all three fixed, none retired	partial: the three doc-gate refusals predate the fix	yes	closed
#8 PreCompact capture	yes, 47 tests	no, 0 rows with source:"precompact" in the queue	yes, payload and cost on 2.1.259	closed; budget and custom_instructions gaps moved to #32
#19 composition and the applied skill	partial	no	no	open, retitled and narrowed

Blocking everything above

CI has been red on main since 2026-08-26, on both matrix legs. #35, P0. A tracker that
closes issues against a red suite is asserting things it cannot check.

What the user asked for

i'm a little surprised that the skills take SO long to build-- i thought a skill was
primarily a markdown file, plus some useful scripts? skills can be relatively simple and
narrowly scoped, as long as they are useful. compounding knowledge can also be as simple
as making a note in CLAUDE.md (the local or global version), or building a searchable
reminder document that gets automatically injected into context at the appropriate times.

That is built. What it has not yet done is produce a skill under the new budget and show it
being used on the problem that caused it.

[jeremymanning]

Wave 1 landed in commit 03773ad (03773addf0d42916fd9303971b764d90e7a59bd6, "Wave 1 of the re-plan: honest instruments, a health surface, and the doctrine in the installer"): 44 files, 1986 tests, 0 failures, 2 documented skips.

What it advanced:

• Measurement fixes: unary counter, skillreport double-count, verdict never wired, lost verdict #24 (measurement fixes): done. Unary counter fixed and skillreport's per-forge double-count deduped.
• skillforge doctor, and a TTL reaper for stuck forges #25 (doctor/reaper): done. skillforge doctor and a TTL reaper for stuck forges shipped.
• Installer writes the compound-improvement doctrine stanza into ~/.claude/CLAUDE.md under a marker #26 (installer doctrine stanza): done. The installer now writes the doctrine block into ~/.claude/CLAUDE.md under a marker.
• Trivia: routing skip text says 54 calls (real: 72); round-cap wording drift across three files #29 (trivia): done. Routing skip text and round-cap wording drift fixed.
• Real conversion rate: 10.5% of nudged sessions invoke skill-compounder #30 (conversion baseline): baseline recorded, not a fix. 10.5% of nudged sessions invoke skill-compounder across all projects.
• Reminder store injected on UserPromptSubmit/PreToolUse (tier 1) #21 (reminder store): partial. Only the PreToolUse measurement is done (a live spike found that imperative wording in injected reminders gets refused as prompt injection, so reminders must be statements of fact). The store itself and the inject hook are Wave 2 work.

Wave 2 (#20 skillnote, #21 reminder store, #22 forge diet, CLI half) is in progress: three agents dispatched on disjoint files.

[jeremymanning]

Wave 2 (7507a0b) and Wave 3 (15b3b28) both landed, closing #20-#23, #27, #28.

Installed for real on this machine: ./install.sh wired 14 hook entries over 8 scripts, 6 CLIs, and the doctrine block came back "already current" on rerun. skillforge doctor: 8 pass, 0 fail. Live end-to-end (CLI 2.1.259, notes/2026-09-02-audit-and-replan.md ~22:05-22:20): tier 0 note read back 3/3; tier 1 UserPromptSubmit 3/3; tier 1 PreToolUse 0/3 on first pass (two skillnote bugs, see #20), 3/3 after the fix.

What remains open:

• PreCompact: capture skill candidates before compaction, with no model in the hook #8 (PreCompact capture): in progress as Wave 4, dispatched alongside this comment round.
• Deeper skill composition, and the last step of the loop: the forged skill actually used #19 point 4 (deeper composition): not started this round.
• Real conversion rate: 10.5% of nudged sessions invoke skill-compounder #30's conversion rate needs re-measurement once the two cheap tiers have accumulated real usage data; skillnote: a CLI for the cheap branch (tier 0) #20 and Reminder store injected on UserPromptSubmit/PreToolUse (tier 1) #21 only just gave them a mechanism to measure.
• The two hooks/remind.sh constants, REMIND_MAX=2 and REMIND_COOLDOWN=0, are unvalidated. bin/skillreport is the instrument that would settle them, and needs real usage across several repositories over real time before either is tuned.

[jeremymanning]

Tracker reconciled on 2026-09-03 against the tree at 385624f. The body's wave checklist is
replaced by a status table, since three of the four waves are committed and a checklist no
longer says what is true.

Closed, each with acceptance evidence read off the code in its closing comment: #8, #20,
#21, #22, #23, #24, #25, #26, #27, #28, #29.

Still open from the original plan: #30 (the measurement campaign, now with the next
sweep's counting rules stated) and #19 (retitled to "Deeper skill composition, and the last
step of the loop: the forged skill actually used", after points 1 and 2 closed and most of
point 3 turned out to be answered by measurement).

New issues

Three are residuals of issues closed above, and are linked from them:

• PreCompact: the 100 ms budget is a system-jq figure, and custom_instructions is unprobed #32 PreCompact: the 100 ms budget is a system-jq figure, and custom_instructions is
  unprobed. From PreCompact: capture skill candidates before compaction, with no model in the hook #8.
• remind.sh: no prune of <state>/remind/, hits.jsonl unbounded on write, REMIND_MAX and REMIND_COOLDOWN unvalidated #33 remind.sh: no prune of <state>/remind/, hits.jsonl unbounded on write,
  REMIND_MAX and REMIND_COOLDOWN unvalidated. From Reminder store injected on UserPromptSubmit/PreToolUse (tier 1) #21.
• No forge has run since the diet landed: round cap, 30-minute target, apply and verdict rows all unexercised #34 No forge has run since the diet landed: round cap, 30-minute target, apply and
  verdict rows all unexercised. From Forge diet: one builder, one cold reviewer, two rounds by default #22 and Measurement fixes: unary counter, skillreport double-count, verdict never wired, lost verdict #24, and the gating fact for Deeper skill composition, and the last step of the loop: the forged skill actually used #19.

Six are new, found while checking the above:

• CI: both matrix jobs red on main #35 CI: both matrix jobs red on main. P0. Every run since 2026-08-26 has failed,
  including the newest at c06eb6c. A triage table follows on that issue.
• E2E certification: one complete user journey in a throwaway Claude config #36 E2E certification: one complete user journey in a throwaway Claude config. Eleven
  steps from fresh install to uninstall. notes/OPEN-THREADS.md:343-351 has been asking for
  this since before the tiers existed.
• Event attribution: stable ids joining candidate, note, reminder, skill, delivery, outcome #37 Event attribution: stable ids joining candidate, note, reminder, skill, delivery and
  outcome. hits.jsonl records a delivery and no outcome, so the only conversion figure in
  the package is reconstructed from edit counters. Blocks Real conversion rate: 10.5% of nudged sessions invoke skill-compounder #30's re-measurement.
• Release packaging: tagged releases, pinned installer, explicit update with rollback #38 Release packaging: tagged releases, pinned installer, explicit update with rollback.
  No tags, no releases, install.sh clones main shallow and updates with pull --ff-only.
• Default-on paid session review should be opt-in for a curl|bash install #39 Default-on paid session review should be opt-in for a curl | bash install.
  hooks/session-review.sh:186 defaults SKILL_COMPOUNDER_REVIEW to 1; the README discloses
  it and the installer does not ask.
• Docs split: README to quickstart, plus docs/architecture, operations, measurement, development #40 Docs split: README to a quickstart plus docs/architecture, operations,
  measurement, development. README.md is 1086 lines.
• Hygiene: skillreport header contradicts the counter format, ShellCheck absent from CI, Actions Node 20 deprecation #41 Hygiene: bin/skillreport:58 still calls the counters integers, ShellCheck is
  absent from CI, and both failing jobs carry a Node 20 deprecation warning. A fourth item,
  Python invalid-escape warnings, was checked and dropped: 55 files parse clean under
  -W error::SyntaxWarning.

Suggested order: #35 first, because nothing else can be certified against a red suite; then
#34, which is the one run that would produce readings for #19, #22 and #30 at once.

[jeremymanning]

v0.3.0 tagged at a2aa2d4 (git ls-remote --tags origin v0.3.0 -> 4eddceb...), release published. CI is green on both matrix legs plus shellcheck and plugin-validate (runs 33719557434, 33726173128). E2E: 12/12 PASS, 6 calls, 41 s, on a2aa2d4 (notes/2026-09-02-audit-and-replan.md ~04:05).

This round closed #35, #36 (opened #42 for the auth-isolation follow-up), #39, #40 (README is 366 lines, not the under-250 the issue stated; closed anyway, noted in its closing comment). #38 and #41 stay open, each scoped narrower than they arrived: #38 to one missing test (--update/--rollback has no automated coverage, only a manual runbook that itself admits rollback-between-tags was never exercised); #41 to one line (bin/skillreport:58 still calls the counter an integer, contradicting :1308).

What remains open on the tracker: #19 (deeper composition, the forged skill actually used), #30 (conversion re-measurement), #32 (PreCompact budget and custom_instructions), #33 (remind.sh pruning and bounds), #34 (no forge has run since the diet landed), #37 (event attribution), #38, #41, and the new #42.

[jeremymanning]

v0.3.1 is tagged at b7f6a47, with CI green on that commit: run 33731348166 (main, conclusion success).

Open issues by number: #9, #19, #30, #32, #33, #34, #37, #38 has just closed, #41 has just closed, #42.

Remaining open after this comment: #9, #19, #30, #32, #33, #34, #37, #42.