Release/0.7.0

.github/workflows/ci.yml

@@ -0,0 +1,24 @@
+name: CI
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+jobs:
+  test:
+    name: Python ${{ matrix.python-version }}
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Run unittest suite
+        run: python -m unittest
+      - name: Demo graph provenance check
+        run: MYGRAPH_PATH=examples/demo_graph.json python -m mygraph.mygraph check --provenance

CHANGELOG.md

@@ -0,0 +1,30 @@
+# Changelog
+
+## 0.7.0
+
+- Added `mykg deep-dive`, a pre-ingest workspace flow for generating adaptive
+  artifacts, validation reports, an artifact-local graph summary, and canonical
+  ingest candidates without mutating `MYGRAPH_PATH`.
+- Added `mykg deep-dive inspect <workspace>` for reviewing generated artifacts,
+  candidate counts, validation status, and the next suggested command.
+- Added `mykg deep-dive add-to-graph <workspace>` as a convenience wrapper over
+  the existing `ingest --candidates-file` validation/review/merge path.
+- Added the Deep-Dive Interaction Model documentation, defining generate,
+  inspect, challenge, add-to-graph, approve, and don't-ingest-yet semantics.
+- Expanded README workflow docs to explain how candidates become durable graph
+  memory and where deep-dive fits before ingest.
+- Updated the public roadmap to make v0.7.0 the interaction-model and
+  deep-dive workspace release, with memory analyzer work tracked next.
+- Added broader CLI regression coverage for help/error paths and the existing
+  query, list, path, check, export, context, viz, audit, discover, ingest, and
+  deep-dive command surfaces.
+- Added GitHub Actions CI for pull requests and main-branch pushes across
+  Python 3.10, 3.11, 3.12, and 3.13, including the unittest suite and demo
+  graph provenance check.
+
+## 0.6.x
+
+- Added memory audit, context export, and discovery-layer workflows over the
+  public demo graph.
+- Kept audit and discover read-only: they rank, explain, and propose, but never
+  mutate graph memory directly.

README.md

@@ -35,6 +35,8 @@ matrix and [Benchmarks](docs/BENCHMARKS.md) for the offline demo-graph checks.
 ## What It Does
 
 - Ingests markdown notes into candidate graph nodes and edges.
+- Generates pre-ingest deep-dive workspaces for sources that need synthesis
+  before graph promotion.
 - Requires provenance excerpts before claims become durable memory.
 - Lets you review, accept, reject, or edit LLM proposals before merge.
 - Searches by term, lists nodes by type, and finds paths between ideas.
@@ -161,6 +163,7 @@ Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
 | `list <type>` | List nodes of a given type |
 | `path <a> <b>` | Find the shortest path between two nodes |
 | `ingest <file.md>` | Extract, validate, review, merge, and eval candidates |
+| `deep-dive <file.md>` | Generate a pre-ingest workspace with artifacts and candidates |
 | `check --provenance` | Flag nodes with missing source citations |
 | `export --ttl` | Emit Turtle/RDF |
 | `context` | Print a compact LLM-ready context snapshot |
@@ -176,6 +179,26 @@ Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
 
 You can ingest your notes with or without an API key.
 
+### How Memory Enters The Graph
+
+Graph memory is promoted through a review lifecycle:
+
+```text
+source note
+  -> candidates.json
+  -> validate
+  -> review
+  -> merge accepted items into MYGRAPH_PATH
+```
+
+Candidates are proposals, not memory. Validation checks schema, IDs,
+provenance excerpts, and edge endpoints. Review is the promotion gate. The
+active graph changes only after accepted candidates are merged.
+
+The product contract is:
+
+> The model proposes. Artifacts expose reasoning. Provenance verifies. Human review promotes.
+
 ### Claude or Codex App, No API Key
 
 If you are already working with Claude, Codex, or ChatGPT in an app session, you do **not** need an API key. Ask the assistant to produce a `*.candidates.json` file that follows the schema in `mygraph/extractor.py`, then let the local CLI validate, review, and merge it. In Claude Code, the bundled [`/ingest-notes`](.claude/skills/ingest-notes/SKILL.md) skill runs this flow for you:
@@ -190,6 +213,28 @@ mykg ingest path/to/your/notes.md --candidates-file path/to/your/notes.candidate
 
 The app subscription helps you create the candidates file. The repo still keeps graph validation and merge local.
 
+### Deep Dive Workflow
+
+Use `deep-dive` when a source needs synthesis, challenge, or a reasoning
+workspace before it becomes graph memory:
+
+```bash
+mykg deep-dive notes.md --out-dir ~/private/deepdives/notes
+mykg deep-dive inspect ~/private/deepdives/notes
+mykg deep-dive add-to-graph ~/private/deepdives/notes
+```
+
+Generation creates a workspace with `manifest.json`, `artifact-plan.json`,
+Markdown artifacts, validation reports, an artifact-local graph summary, and
+canonical candidates. It does **not** mutate `MYGRAPH_PATH`.
+
+`add-to-graph` reads the workspace manifest and delegates to the existing ingest
+validation/review/merge path. Keep using `ingest` directly when you already have
+a focused source note or hand-curated candidates file.
+
+See [Deep-Dive Interaction Model](docs/DEEP_DIVE_INTERACTION_MODEL.md) for the
+generate, inspect, challenge, approve, and add-to-graph semantics.
+
 ### Automated API-Backed Ingest
 
 If you want the CLI to call an LLM directly, use a provider API key or local Ollama.

docs/DEEP_DIVE_INTERACTION_MODEL.md

@@ -0,0 +1,104 @@
+# Deep-Dive Interaction Model
+
+`knowledge-worker` has one durable memory rule:
+
+> The model proposes. Artifacts expose reasoning. Provenance verifies. Human
+> review promotes.
+
+This document defines the product language around the current ingest flow and
+the `deep-dive` workflow added in v0.7.0.
+
+## Current Memory Lifecycle
+
+Today, durable memory enters the graph through ingest:
+
+```text
+source note
+  -> candidates.json
+  -> validate
+  -> review
+  -> merge accepted items into MYGRAPH_PATH
+```
+
+Candidates are proposals. Validation checks shape, IDs, allowed node/edge
+types, endpoint references, and high-confidence excerpts. Review decides what
+gets promoted. Merge writes only approved material to the active graph.
+
+## Deep-Dive Lifecycle
+
+`deep-dive` adds a reasoning workspace before ingest:
+
+```text
+source
+  -> deep-dive workspace
+  -> artifacts
+  -> evidence validation
+  -> refinement
+  -> candidates.json
+  -> validate
+  -> review
+  -> merge accepted items into MYGRAPH_PATH
+```
+
+In v0.7.0 the local generator creates conservative starter artifacts and
+validated canonical candidates. Future LLM-backed generators can deepen the
+workspace without changing the state model.
+
+## User Intent Semantics
+
+`generate`
+: Create a workspace with artifacts, manifest, validation report, artifact
+graph summary, and optional canonical candidates. This never mutates
+`MYGRAPH_PATH`.
+
+`inspect`
+: Summarize the workspace: artifacts, source path, candidate counts, validation
+status, weak spots, and next command.
+
+`challenge this`
+: Critique or refine the workspace. This is not rejection and not graph merge.
+It means the reasoning surface needs another pass.
+
+`add to graph`
+: Start the validation/review/merge path. In v0.7.0 this is implemented as
+`mykg deep-dive add-to-graph <workspace>`, which delegates to existing ingest.
+
+`approve X`
+: Promote only selected material where the review path supports selection.
+Approval is a graph-memory action, not an artifact-generation action.
+
+`don't ingest yet`
+: Keep outputs artifact-local. The workspace can still be useful without
+becoming durable graph memory.
+
+## Workspace Contract
+
+Every generated workspace includes:
+
+- `manifest.json`: source path, artifact list, candidate path, validation
+status, mutation flag, and next suggested command.
+- `artifact-plan.json`: the source profile, chosen artifacts, and acceptance
+criteria.
+- Markdown artifacts: reviewable reasoning surfaces with source-local evidence
+references.
+- `validation-report.json`: canonical candidate validation summary when
+candidates are generated.
+- `artifact-graph.json`: artifact-local summary, not canonical graph memory.
+- `*.candidates.json`: ingest-compatible proposals when candidates are enabled.
+
+## Known Review Limitation
+
+Current interactive ingest reviews candidate nodes directly. Eligible edges are
+included after node approval when their endpoints are approved or already exist.
+That keeps ingest short, but edges are reasoning claims and deserve better
+review ergonomics. Improved edge review remains a roadmap item.
+
+## Safe Mental Model
+
+```text
+Artifacts are thinking.
+Candidates are proposals.
+Validation is evidence discipline.
+Review is promotion.
+The graph is accepted memory.
+```

docs/ROADMAP.md

@@ -3,35 +3,52 @@
 ## v0.3: Directed Memory Audit And Benchmarks (shipped)
 
 - Emit `analytics.json` with PageRank, betweenness, k-core, communities,
-  low-confidence edges, and provenance coverage (`mykg audit`).
+  low-confidence edges, provenance coverage, and directed idea-flow queues
+  through `mykg audit`.
 - Add a Memory Audit HTML view with ranked panels before the graph canvas:
-  important concepts, bridge ideas, weak claims, and proof trail.
-- Add directed idea-flow panels that separate idea attractors from idea
-  generators.
-- Turn weak claims into a user-reviewed queue: verify, downgrade, convert to a
-  question, or ignore for now.
-- Keep the audit read-only and prompt-driven so the user does the judgment work.
-- Publish `docs/COMPETITIVE_ANALYSIS.md` with a source-checked category matrix.
-- Publish `docs/BENCHMARKS.md` with offline demo-graph benchmarks.
-- Add `tests/test_benchmarks.py` so benchmark checks run with no API key.
-- Add README positioning that points readers to the analysis and benchmarks.
-
-## v0.4: Discovery Layer (this release)
+  important concepts, bridge ideas, weak claims, proof trail, and legwork
+  queues.
+- Publish `docs/COMPETITIVE_ANALYSIS.md` and `docs/BENCHMARKS.md`.
+- Add offline benchmark tests over `examples/demo_graph.json`.
+
+## v0.4: Discovery Layer (shipped)
 
 - Add `mykg discover`: derived-edge proposals and second-order analytics on top
-  of the audit layer — staleness radar, co-mention inference, goal-alignment
-  candidates, question debt, corroboration scoring, de-spined bridge detection,
-  and tension detection.
-- Keep every inference a *proposal*: discover never mutates the graph; output
-  goes to a promotion queue for human review.
-- Expand the fictional demo graph to launch scope (multiple communities, bridge
-  ideas, low-confidence candidate edges) and commit generated demo analytics.
+  of the audit layer.
+- Keep every inference a proposal: discover never mutates the graph; output goes
+  to a promotion queue for human review.
+- Expand the fictional demo graph so discovery has stale, weak, bridge, and
+  tension examples to surface.
+
+## v0.5-v0.6: Packaging And Context Surfaces (shipped)
+
+- Package the CLI as `knowledge-worker` with `mykg` and `mygraph` entrypoints.
+- Keep the core graph CLI stdlib-only, with optional extras for RDF and LLM
+  extraction backends.
+- Support context export, offline visualization, local Ollama adapter surfaces,
+  and public-demo-safe graph fixtures.
+
+## v0.7: Deep-Dive Interaction Model And Workspace
+
+- Add `mykg deep-dive` as a pre-ingest workspace generator:
+  artifacts, artifact plan, manifest, validation report, artifact-local graph
+  summary, and canonical candidates.
+- Add `mykg deep-dive inspect <workspace>` for reviewable workspace summaries.
+- Add `mykg deep-dive add-to-graph <workspace>` as a wrapper over existing
+  ingest validation/review/merge.
+- Document the product interaction model: generate, inspect, challenge,
+  add-to-graph, approve, and don't-ingest-yet.
+- Keep `ingest` as the canonical graph mutation path.
 
-## v0.5: MCP Surface Hardening
+## v0.8: Memory Analyzer Layer
 
-- Complete and document the local MCP wrapper surface.
-- Add MCP smoke tests that do not require private graph data.
-- Revisit named competitor rows only after a fresh source-verification pass.
+- Combine audit, context export, discovery, and deep-dive outputs into a single
+  user-facing memory analyzer report.
+- Summarize what the graph knows, what it can prove, what it should review, and
+  which candidates are ready for promotion.
+- Improve edge-review ergonomics so relationships can be accepted, rejected, or
+  edited as first-class reasoning claims.
+- Add evals for analyzer usefulness on the public demo graph.
 
 ## v1: Public Demo And Local Graphs
 
@@ -40,20 +57,8 @@
 - Generate offline HTML viewers with embedded graph JSON.
 - Keep provenance checks at zero violations.
 
-## v1.5: Better Review Loops
-
-- Improve candidate review ergonomics, including the discover promotion queue.
-- Record clearer eval outcomes for accepted, rejected, and edited claims.
-- Add repeatable privacy scans to the normal commit checklist.
-
 ## v2: Storage Evolution
 
 - Move from JSON to SQL-backed persistence only when graph size or concurrency
   makes JSON awkward.
 - Preserve the public node/edge schema and CLI behavior.
-
-## v3: Productization
-
-- Package the CLI for repeatable installation.
-- Add import/export recipes for common note-taking and AI-export formats.
-- Document deployment patterns that keep private graph data local by default.