Skip to content

Event attribution: stable ids joining candidate, note, reminder, skill, delivery, outcome #37

Description

@jeremymanning

The package now has six artifact kinds and no way to follow one thing across them. Every
number that spans two of them is reconstructed by counting, and a count cannot say which
delivery produced which outcome.

Evidence

The delivery log carries an id and stops there. hooks/remind.sh:432 appends one row
per delivery. The first three rows of
~/.claude/skill-compounder/remind/hits.jsonl:

{"id":"n2594918537x212","ts":1788390353,"session":"80fb75d1-fa2a-4de5-8a6c-95e65aff865d","event":"UserPromptSubmit"}
{"id":"n2594918537x212","ts":1788390360,"session":"9ad6f1b8-7b94-412e-b66a-0599a655d869","event":"UserPromptSubmit"}
{"id":"n2594918537x212","ts":1788390365,"session":"7059f6ce-fea9-4021-a467-a5d44eab65c1","event":"UserPromptSubmit"}

Four fields: which reminder, when, which session, which event. The row says a reminder was
delivered. Nothing in it, or joinable to it, says whether the session then did the thing.
There is no outcome field and no back-reference from any later row to a hit.

So the one conversion figure in the package is reconstructed from counters.
bin/skillreport:1300-1334 computes REMINDER CONVERSION by reading per-session *.edits
tallies, dividing each by CI_EDIT_EVERY to guess how many checkpoints a session must have
hit, and comparing that to the all-time count of start rows. It prints its own caveat:

REMINDER CONVERSION (estimate, see the comment in this script)
  edit counters on disk:      115 session(s)
  checkpoints they imply:     376 (assuming CI_EDIT_EVERY=12)
  forges started, all time:   10
  rough conversion:           2%
  The numerator covers all time and the denominator only the last 7 days, so read
  the conversion as a loose upper bound rather than a rate.

The comment at bin/skillreport:1302 says why: "The reminder hook does not log when it
fires, only the running edit count per session."

The other links are the same shape. A note row and a reminder row share an id
(skillnote --remind writes both), and 18 note rows exist against 16 hits, but no ledger
row records that a note was read back. origin connects a skill to a trigger string, not to
the queue entry that produced it. The chain candidate to note to reminder to skill to
delivery to outcome is six hops and three of them are unjoinable.

What to build

One id, minted at capture, carried on every row that descends from it, in every store:
insight queue, reminders.jsonl, hits.jsonl, the ledger's note, origin, use,
apply and verdict rows.

Then a funnel is a join rather than an estimate: delivered N times, acted on M, outcome
recorded K. That also settles what #30 asks for and what .claude/CLAUDE.md says
bin/skillreport is the instrument for.

Constraint from .claude/CLAUDE.md: every ledger reader selects its events by name, so
adding a field is safe and widening a selector to a negation is not.

Acceptance

  • A test writes a candidate, promotes it, delivers it, records an outcome, and reads one id
    back from all six stores.
  • bin/skillreport reports the funnel as counted rows, and says so, rather than as an
    estimate with a caveat.
  • Existing rows without an id are reported as unattributed rather than dropped or guessed
    at, the way trigger_kind:"unrecorded" already handles a missing trigger.

Blocks the re-measurement #30 asks for.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions