Real conversion rate: 10.5% of nudged sessions invoke skill-compounder

[jeremymanning]

Recording the one number that says whether the reminder path works, so the three tiers have
something to beat.

The measurement

Agent 5 of notes/2026-09-02-audit-and-replan.md swept 1456 transcripts across every
project on this machine:

figure	value
sessions where the checkpoint or prompt hook fired	866
sessions that invoked skill-compounder	96
sessions that did both	91
conversion	10.5%

This project alone: 249 nudges, 3 invocations.

So roughly nine in ten nudged sessions read the reminder and carry on. That is the
denominator #19 is about: recognition during work is absent, and a reminder injected into
a thread that is deep in another task is mostly not acted on.

What to do with it

1. Store the sweep as a script under scripts/, so the figure is re-derivable rather than
   quoted. A number nobody can re-run is not a baseline.
2. Re-run it after tier 0 and tier 1 land. A note takes seconds and a reminder arrives
   keyed to the moment, so both should convert better than 10.5% or the tiers have not
   earned their place.
3. Report per project as well as overall. 3 of 249 here against 91 of 866 everywhere
   suggests the rate depends on what the session is doing, and one aggregate hides that.

What this does not settle

.claude/CLAUDE.md records the 12-edit and 20-minute hook constants as unvalidated, and
names bin/skillreport as the instrument that would settle them. That instrument cannot
compute its conversion figure at all right now: the hook writes a unary tally the reader
rejects as non-numeric. See the measurement-fixes issue. Until that is fixed, this
transcript sweep is the only conversion number available, and it says nothing about which
edit count is right.

Do not tune the constants against this figure.

Wave 1 of the three-tier epic. Depends on the measurement-fixes issue for anything beyond
the raw rate.

[jeremymanning]

Staying open as the measurement campaign. Two things have changed since the 10.5% was
recorded, and both change what the next measurement has to count.

The tiers this figure was a baseline for now exist

Verified today at 385624f:

• Tier 0, bin/skillnote, shipped in 7507a0b. 18 note rows in
  ~/.claude/skill-compounder/ledger.jsonl where the ledger held none.
• Tier 1, hooks/remind.sh, shipped in 7507a0b, wired on UserPromptSubmit and on
  PreToolUse for Bash|Write|Edit in both hooks/hooks.json and the installed
  ~/.claude/settings.json. 16 delivery rows in
  ~/.claude/skill-compounder/remind/hits.jsonl.
• Tier 2, the forge diet, in 7507a0b and 15b3b28.

So the second item in this issue's body, re-run it after tier 0 and tier 1 land, is now
runnable. It has not been run.

The 10.5% measures one thing, and the next measurement must not reuse it

That figure is nudge-to-skill: of 866 sessions where the checkpoint or prompt hook fired, 91
invoked skill-compounder. It counts one path from one nudge to one skill invocation. The
package now has three delivery paths with different shapes, and folding them into a single
percentage would hide exactly the difference the tiers were built to produce.

The next sweep counts these separately:

1. Note read-backs. A note is passive: written into a marker block, read on the next
   session's context load. Success is a later session acting on the line, not invoking
   anything. The read-back mechanism is measured (docs/CLAUDE-CODE-BEHAVIOR.md, CLI 2.1.258:
   MEMORY.md is injected with no tool call 3/3, a file the index links to is fetched with a
   Read 3/3, an unindexed file in the same directory 0/3), so a scope that is not read back
   is a known condition rather than a null result.
2. Reminder deliveries. Count the rows in
   ~/.claude/skill-compounder/remind/hits.jsonl, per event arm, since UserPromptSubmit and
   PreToolUse were separately measured at 3/3 and 0/3-then-3/3 and may not behave alike.
3. Hits, meaning what a delivery led to. This is the one with no instrument.
   hits.jsonl carries {"id","ts","session","event"} and no outcome field, and
   bin/skillreport's REMINDER CONVERSION reconstructs from edit counters rather than
   joining rows, printing its own caveat that "the numerator covers all time and the
   denominator only the last 7 days". Event attribution: stable ids joining candidate, note, reminder, skill, delivery, outcome #37 is the attribution work that would make this a
   join.
4. Nudge to skill, the original 10.5%, kept as the comparable series.

Per project as well as overall, per item 3 of the body: 3 of 249 here against 91 of 866
everywhere.

Still standing from the body

The sweep is not yet a script under scripts/, so the figure is quoted rather than
re-derivable. That is item 1 of this issue and it is the first thing to do.

Do not tune CI_EDIT_EVERY or CI_PROMPT_COOLDOWN against any of this, per
.claude/CLAUDE.md. The same restraint now applies to REMIND_MAX and REMIND_COOLDOWN
(#33).