feat: detect and reconcile docs created outside the API#32
Open
wormuz wants to merge 6 commits into
Open
Conversation
added 6 commits
May 6, 2026 11:50
Files written under ~/.opencontext/contexts/ via Write/Edit (bypassing
oc_create_doc) used to silently disappear from oc_manifest and oc_search
because the manifest was a pure SELECT from SQLite. Agents thought the
folder was empty and proposed re-creating it.
Three changes that make the drift impossible to miss:
1. oc_manifest now scans the filesystem after the SELECT and returns
{ items, unindexed_files }. When unindexed_files is non-empty, MCP
surfaces a `warning` field naming each file. Manifest stays read-only
- no auto-insert.
2. New `oc index reconcile [folder]` CLI command and `oc_reconcile_folder`
MCP tool. They INSERT a docs row for every *.md present on disk but
missing from SQLite, without recomputing embeddings (run
`oc index build` separately for that). Fast path for picking up
orphans after a Write or `git pull`.
3. oc_create_doc / oc_set_doc_desc descriptions now open with a hard
"REQUIRED for any file under ~/.opencontext/contexts/" notice that
explicitly calls out Write/Edit as a bug, so MCP-aware agents stop
reaching for them.
Rust core gains generate_manifest_full() and reconcile_folder() (the
old generate_manifest is preserved for Tauri/UI/CLI callers). NAPI
binding switches generateManifest to the wrapped form; existing JS
callers updated. Three new core tests cover orphan detection,
nested-folder scan, and reconcile idempotency. Tested end-to-end on a
real folder with 6 orphans - manifest reported drift, reconcile
registered all 6, follow-up manifest came back clean.
The previous commit's oc_create_doc / oc_set_doc_desc descriptions told agents to call `oc_save_doc` for body edits, but that tool was never registered in src/mcp/server.js — only the Rust core / NAPI / JS layers had it. Agents had to fall back to Edit on the absolute path, which is safe but contradicts our own "REQUIRED — never use Write/Edit" guidance and skips the updated_at bump + search-sync event. Register oc_save_doc as a thin wrapper over store.saveDocContent so the description's promise is actually fulfillable. Tighten neighbouring descriptions to mention only tools that exist (drop the phantom `oc_update_doc` and the JS-internal `saveDocContent`).
clippy::ptr_arg with -D warnings treats &PathBuf args as errors. Replace with &Path — callers pass &path (PathBuf) which coerces automatically.
oc_search matches document body only — not folder/file names. Agents searching by project name got 0 results and wrongly concluded the project doesn't exist. Add explicit routing hint to both descriptions: - oc_search: 'Does NOT match folder/file names — use oc_manifest for known projects' - oc_manifest: 'Use this when you know the project/folder name — oc_search matches content only'
- cargo fmt: fix unindexed_files formatting and use std import style to match CI rustfmt version - cargo fmt: split multi-line name chain and condense write calls in tests - test-js: uninstall published @aicontextlab/core-native before running tests so the locally built binary is loaded instead of stale npm package
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Files written under ~/.opencontext/contexts/ via Write/Edit (bypassing oc_create_doc) used to silently disappear from oc_manifest and oc_search because the manifest was a pure SELECT from SQLite. Agents thought the folder was empty and proposed re-creating it.
Three changes that make the drift impossible to miss:
oc_manifest now scans the filesystem after the SELECT and returns { items, unindexed_files }. When unindexed_files is non-empty, MCP surfaces a
warningfield naming each file. Manifest stays read-onlyNew
oc index reconcile [folder]CLI command andoc_reconcile_folderMCP tool. They INSERT a docs row for every *.md present on disk but missing from SQLite, without recomputing embeddings (runoc index buildseparately for that). Fast path for picking up orphans after a Write orgit pull.oc_create_doc / oc_set_doc_desc descriptions now open with a hard "REQUIRED for any file under ~/.opencontext/contexts/" notice that explicitly calls out Write/Edit as a bug, so MCP-aware agents stop reaching for them.
Rust core gains generate_manifest_full() and reconcile_folder() (the old generate_manifest is preserved for Tauri/UI/CLI callers). NAPI binding switches generateManifest to the wrapped form; existing JS callers updated. Three new core tests cover orphan detection, nested-folder scan, and reconcile idempotency. Tested end-to-end on a real folder with 6 orphans - manifest reported drift, reconcile registered all 6, follow-up manifest came back clean.