feat(daemon): reflective abstraction — aha-gated abstract_entries action#19
Merged
Conversation
…tion) Add a new structural action that synthesises a higher-order PARENT entry abstracted from >=2 related leaf entries (bottom-up schema formation), mirroring the merge_entries / split_folder template end-to-end: - _schemas.py: AbstractEntries model (child_paths min-2 validator, parent_path/title/body) added to BOTH the Librarian ProposedAction union and the Scholar ScholarAction union; ScholarAbstractEntries subclass holds parent_body to the frontmatter bar (type: abstraction). - scholar.py: wire abstract_entries into _action_body_and_path (parent body wikilink validation), _created_paths (parent is batch-created), _apply_count_deltas (parent +1 to its dir; children unchanged), plus a new _validate_abstract_children_exist whole-batch check (children must already exist on disk). - librarian.py: parent_path + each child_path validated as well-formed .md inside the knowledge root; parent_body frontmatter must parse. Path-field extraction refactored to data tables to keep complexity under the lint cap. - scholar_executor.py: _do_abstract writes the parent, upserts index.md, appends log.md, and adds an idempotent reverse [[parent]] backlink into each child's ## Related section. Children stay in place (not archived, not moved); an unreadable child is noted, not fatal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add abstract_entries to the action vocabulary in both prompts, gated on a genuine "aha" (the same surprise bar leaf writes use), so the model only proposes/approves predictive abstractions, never topical groupings: - librarian_prompt.py: propose-side guidance with the four-part gate (remote children, predictive lift, non-obvious, compresses), the GOOD example (likes-lint-in-python + likes-lint-in-js -> "likes linting across languages", predicts Rust), and explicit MUST-REJECT negatives (same-surface "yellow things", vague "fast tools", worthless "good code"). Stresses: propose rarely. - scholar_prompt.py: expand the returnable vocabulary from six to seven actions; add an ABSTRACTION GATE section instructing a precision VETO for proposals that fail the aha bar (vague / obvious / non-predictive / same-domain / premature / duplicate parent), with the same examples. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ompts - schema: validates a well-formed abstraction; rejects <2 children, empty children, unparseable/empty/id-mismatched parent_body; union discrimination. - scholar validators: passes when children exist; raises on a missing child; parent registered as a created path; parent counts toward the flat-dir cap. - librarian validators: well-formed parent/child paths pass; bad child path and unparseable parent_body raise. - executor: writes a type:abstraction parent, backlinks children (append new ## Related and append-to-existing branches), idempotent re-run, index/log maintained, children intact, unreadable-child non-fatal; plus direct _add_parent_backlink edge cases. - prompts: assert the aha/abstraction gate text + GOOD/MUST-REJECT examples render in both assembled prompts. Total coverage 90.68% (>= 90% gate); 609 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the speculative prose + mechanisms-table TODO with the implemented surprise-/aha-gated abstract_entries design (four-part gate on both propose- and veto-side; deterministic executor writing a type:abstraction parent with child backlinks; children stay in place). Add the neuro grounding: van Kesteren et al. 2012 (SLIMM), Jung-Beeman et al. 2004 (insight = remote association), Kizilirmak et al. 2016 (aha = reward-PE), Schlichting & Preston 2015 / Tse et al. 2007 / Behrens et al. 2018 (integration -> novel inference). Note that encoding_strength-derived parent durability remains a follow-up (depends on the encoding_strength field); this ships on the existing reinforced/decay lifecycle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements reflective abstraction (roadmap's largest open gap): a new curator action
abstract_entriesthat synthesizes a higher-order parent rule from ≥2 related leaf entries. Children stay individually retrievable; the parent links them via[[wikilinks]]. Bottom-up schema formation — not dedup (merge) or reorg (split_folder).The aha gate (the heart)
Per the neuro scoping, an abstraction is proposed/approved only if it's a genuine "aha" — surprising, non-trivial, predictive. Four criteria (Librarian propose-side + Scholar veto-side):
✅
likes lint (python)+likes lint (js)→ "likes linting across languages" (predicts wanting lint in Rust).❌ rejected: "these are all yellow" / "likes uv + likes ruff → likes fast tools" / "likes lint + likes types → likes good code" — baked in as MUST-REJECT examples.
Neuro grounding in the roadmap: van Kesteren SLIMM (only the surprising/congruent extremes encode — the "yellow" middle doesn't), Jung-Beeman (insight = remote association), Kizilirmak (aha = reward-PE → durable encoding), Schlichting & Preston / Tse / Behrens (integration → novel inference).
What changed
_schemas.py(AbstractEntries+ScholarAbstractEntries, ≥2-children validator, both unions) ·scholar.py/librarian.py(validators: well-formed paths, parseable parent frontmatter, children-must-exist) ·scholar_executor.py(_do_abstract: writes thetype: abstractionparent, syncs index/log, adds an idempotent child→parent## Relatedbacklink; children otherwise untouched; unreadable child is integrity-first) ·librarian_prompt.py/scholar_prompt.py(the aha gate) ·README.mdroadmap (TODO → "Done, slice 1") · tests across 6 files.Review (done before this PR)
pydantic_ai/ANTHROPIC_API_KEY;typed_agent.py+llm.pyuntouched (recursion guard +setting_sources=[]isolation preserved)._do_abstractis dispatched; write-path is scoped toknowledge_dir, backlinks idempotent, children intact.ScholarDecisions/LibrarianProposalschemas generate cleanly (8$defs,abstract_entriespresent) — the bigger union doesn't break schema-gen.Built by a background agent, reviewed line-by-line by me (executor write-path, union wiring, prompt gate, constraints).
Not yet verified (the units can't)
The live MCP-hop with the 7-action schema, and the gate's real-world quality (does the model actually propose good ahas and veto the "yellow" class). Recommend a watched live batch after merge — or I can run an isolated Scholar smoke against a throwaway library first. Fails safe regardless: a bad/oversized schema or a failed run raises
TypedAgentError→ batch dropped (visible in logs), never a loop, data-loss, or billing change.Follow-up (noted in roadmap):
encoding_strength-derived parent durability — depends on the separateencoding_strengthfield; this ships on the existingreinforced/decay lifecycle.🤖 Generated with Claude Code