Every piece of the loop has been proven in isolation. Nothing has proven the loop.
Why this is still open
notes/OPEN-THREADS.md:343-351, under "Still wanted: end-to-end testing the way a user
meets it", already asks for it and says what is missing:
Partly addressed by edc2f60 ("Test the package as a user meets it, and fix what that
found") and by the live routing probes, but not finished; this is issue #10. The remaining
gap: install the package into a throwaway config directory, run real claude -p sessions
against it, and watch the state files, the ledger and the weekly queue while they run,
evaluating the outputs critically rather than checking exit codes. Never against the real
~/.claude.
Since that was written the package gained three tiers, a promotion path, a PreCompact arm
and a doctor, and the pieces were each measured on their own: tier 0 read back 3/3, tier 1
UserPromptSubmit 3/3 (#20, #21). What no run has produced is one candidate travelling the
whole way from capture to a recorded outcome.
The scenario
Eleven steps, in order, in a throwaway CLAUDE_CONFIG_DIR, never against the real
~/.claude:
- Fresh install into the throwaway config.
skillforge doctor passes.
- An ordinary session runs against it and completes with no interference from any hook.
- A note is created with
skillnote add, and a later session reads it back.
- A reminder fires on each of its three match kinds: a prompt keyword, a tool path, a
command signature.
- A candidate is captured at
Stop, and a second at PreCompact.
- That candidate is promoted with
skillinsight promote.
- A narrow skill is forged from it, inside the target time.
- The skill routes in a fresh session, from a prompt that names neither the skill nor the
package.
- The skill is applied to the original problem.
- The
apply, use and verdict rows for it all appear in bin/skillreport.
- Uninstall restores the config:
settings.json byte-identical apart from our own
entries, the status line back to whatever it was, runtime state intact.
What makes this different from the tests
run_tests.sh runs each piece against a temp state root with a pinned clock. This runs the
installed package against a live CLI and reads the artifacts as a user would, which is what
found the two bin/skillnote bugs on #20 after every unit test was green.
Acceptance
- A script under
scripts/, so the run is repeatable rather than described.
- Every step either passes with the artifact quoted, or fails with what it produced instead.
- The run refuses to start against a
CLAUDE_CONFIG_DIR that is the real one.
- The result is recorded with the CLI version, since a routing reading is a reading at a
moment and not a property.
Related: #10, #14, #15, and the reading #34 asks for at step 7.
Every piece of the loop has been proven in isolation. Nothing has proven the loop.
Why this is still open
notes/OPEN-THREADS.md:343-351, under "Still wanted: end-to-end testing the way a usermeets it", already asks for it and says what is missing:
Since that was written the package gained three tiers, a promotion path, a PreCompact arm
and a doctor, and the pieces were each measured on their own: tier 0 read back 3/3, tier 1
UserPromptSubmit3/3 (#20, #21). What no run has produced is one candidate travelling thewhole way from capture to a recorded outcome.
The scenario
Eleven steps, in order, in a throwaway
CLAUDE_CONFIG_DIR, never against the real~/.claude:skillforge doctorpasses.skillnote add, and a later session reads it back.command signature.
Stop, and a second atPreCompact.skillinsight promote.package.
apply,useandverdictrows for it all appear inbin/skillreport.settings.jsonbyte-identical apart from our ownentries, the status line back to whatever it was, runtime state intact.
What makes this different from the tests
run_tests.shruns each piece against a temp state root with a pinned clock. This runs theinstalled package against a live CLI and reads the artifacts as a user would, which is what
found the two
bin/skillnotebugs on #20 after every unit test was green.Acceptance
scripts/, so the run is repeatable rather than described.CLAUDE_CONFIG_DIRthat is the real one.moment and not a property.
Related: #10, #14, #15, and the reading #34 asks for at step 7.