Repository navigation
Thinking record on runs, and a generated Fabric pattern catalog with a suggester - #2816
glitchking1337 wants to merge 1 commit into
Conversation
…a suggester Per-feature Thinking: line in the ISA (ISAFormat), LIFEOS/ROUTER/ThinkingCheck.ts (line, check, fit), HookBridge thinking verbs, LifeOSHooks draws the line beside the route line, Socrates thinking adapter and seed question, Algorithm 8.46.0 § Spend rule. skills/Fabric: PatternCatalog.ts generates Patterns/catalog.yaml (purpose, triggers, input, output, ported intent, hash), SuggestPattern.ts ranks the top three for a plain request, CheckPatterns runs the catalog check, ExecutePattern and UpdatePatterns read the catalog, FabricSystem docs. Tests: 42 across four files. The catalog in this commit was built against this branch's own Patterns/ with --no-model.
|
Note 🤖 Kai (AI assistant) responding on behalf of Daniel Miessler Thanks for the care in this one. The tests, the fail-open callers and the split offer all came through. I'm going to decline both halves, and here's why. The thinking record: on 2026-11-10 we deliberately took out the machinery that planned, drew and checked which thinking skills a run used. That included the per-prompt judges, the route checks and the thinking heads. A per-run record plus a judge for every skip was exactly the kind of scaffolding we decided a capable session model doesn't need. What replaced it is one line in Routing preferences: before real work, look at every skill you have, thinking skills included, and use what helps without being asked. We measure whether that's happening from the transcripts directly, not from a line the run writes about itself. Also, The Fabric catalog and suggester: same reasoning. A generated catalog, a ranker and a shadow model call to pick among about 270 patterns is a skill-router. The model already reads One real find in here: the |
Summary
Two additions, both records rather than gates:
1. The
Thinking:line, a run's written thinking plan. PAI v2.5's THINK phase walked six thinking tools on every request and wrote INCLUDE or EXCLUDE with a reason. LifeOS v8.0 cut that table as scaffolding, and nothing afterwards recorded which thinking skills a run ran or skipped. This brings the record back the way theLane:line did for dispatch:Thinking: <Skill>·<light|full>[, …] ; skipped <Skill>=<phrase>orThinking: none=<phrase>line per feature block, afterLane:. Additive; an ISA without it reads as before.LIFEOS/ROUTER/ThinkingCheck.ts(new, 18 tests):linedraws🧠 THINKING ▸ F1 RedTeam·light (skipped Council) · F2 none;check --sessioncompares the plan with the Skill tool calls in the session transcript and its subagent sidechains;fitasks one shadow Assay question per skipped skill (callerthinking-fit, fail-open) and says once when the engine is unavailable. Every subcommand exits 0.thinkingandthinking-fitverbs; the LifeOSHooks plugin draws the line beside the route line (also when the ISA has noLane:line).thinkingadapter (new, 5 tests): the weekly share of skips judged would-have-changed, unknown under 10 judged runs; one seed question inseeds/template.json.2. A generated Fabric pattern catalog and a suggester. Upstream pattern files carry no metadata and are overwritten on every sync, so metadata lives in a generated sidecar,
skills/Fabric/Patterns/catalog.yaml, never in asystem.md:skills/Fabric/Tools/PatternCatalog.ts(new, 10 tests):buildderives entries frompattern_explanations.mdand each pattern's IDENTITY and OUTPUT sections, with one cached model pass only for a pattern no explanation line covers;checkfails on a missing entry or a moved hash and is wired intoCheckPatterns.tsand the skill's deterministic evals;triggersregenerates the USE WHEN list on request (--write, opt-in).skills/Fabric/Tools/SuggestPattern.ts(new, 9 tests): top three patterns for a plain request, trigger and keyword ranking with a partial-name penalty and a ported-pattern edge, then one shadow Assay question over the shortlist (callerpattern-suggest, fail-open, skipped whenever a local pattern is shortlisted). An explicitly named pattern or intent makes no model call.Pattern: <name> (<reason>)before the pattern runs. UpdatePatterns rebuilds the catalog after every sync and in its re-scrub recovery. Local patterns get their own catalog underUSER/CUSTOMIZATIONS/SKILLS/Fabric/, merged at read time, never written into the shipped file.Notes for the maintainer
Patterns/withPatternCatalog.ts build --no-model; after any pattern change, run the same build orCheckPatterns.tsgoes red (by design: the signal to rebuild).thinking-fit,pattern-suggest) are not added toLifeOS/install/USER/CONFIG/assay-callers.jsonhere because this PR touches system trees only; add them to the template on port, or they run unregistered (shadow, fail-open).pattern_explanations.mddescribescreate_quizas a reading plan, so that catalog purpose follows the wrong line; its triggers still route "create a quiz".Tests
42 of 42 across
LIFEOS/ROUTER/ThinkingCheck.test.ts,LIFEOS/SOCRATES/adapters/thinking.test.ts,skills/Fabric/Tools/PatternCatalog.test.ts,skills/Fabric/Tools/SuggestPattern.test.ts, run inside a fresh clone of main with this branch applied;PatternCatalog.ts check --completeandCheckPatterns.tsexit 0 there.