feat(connector): Obsidian vault data-source connector (cognee#4725)

[tonydzi]

Obsidian vault data-source connector → cognee-community

Implements topoteretes/cognee#4725 (hackathon connector push) under
packages/connector/obsidian/, following the shape of the existing connectors: a
document-mode dlt resource handed straight to cognee.remember(...) — the notion
connector's document routing (DOCUMENT_SOURCE_ATTR, cognee ≥ 1.4.0) combined with the
google-drive/confluence incremental model (write_disposition="merge", per-resource
state cursor, _deleted hard-delete tombstones reconciled by the existing
orphan_cleanup). No core seam, no new cleanup path.

Design as claimed in the issue thread (topoteretes/cognee#4725 (comment)) — built from operating this exact source in production (our lab's fleet runs agents against a shared Obsidian vault daily), not from reading the docs.

Acceptance criteria

• Connects via local filesystem and selects what to ingest — obsidian_source(vault_path) (or OBSIDIAN_VAULT_PATH); exclude=[...] glob patterns scope it further. No auth, read-only.
• Selected content is ingested and searchable — every *.md; YAML frontmatter → record metadata (a metadata JSON column, also folded into the note text as a Properties section so it survives entity extraction); body → content; notes flow through normal cognify as documents.
• Wikilinks mapped to edges, not discarded — all three forms plain markdown parsers miss: [[note|alias]] (target is note, not the alias), [[note#heading]] (target is still note), and embeds ![[note]] (edge + transclusion flag). Resolution is name-based and vault-wide like Obsidian itself (case-insensitive, shortest path on ambiguity, [[dir/note]] path links resolve by path); the link index is built before any row is emitted. Resolved links ship as a structured links JSON column and as explicit "Links to / Embeds" statements in the note text — which is what turns them into note→note edges in the extracted graph.
• Incremental sync picks up only what changed — per-file (mtime, size, sha256) cursor in dlt resource state; unchanged notes are skipped without being read; a changed note re-emits exactly one row.
• Deleting upstream removes it from the graph on next sync — previous run's path snapshot in the same state; absent paths become _deleted hard-delete tombstones, dropped by dlt on merge and purged by orphan_cleanup. A rename is delete + create by construction (Obsidian has no stable note id — path is identity; the README says so rather than pretending otherwise). A zero-note walk over a previously populated vault skips deletion for that run (transient-walk guard, mirroring the confluence connector) instead of tombstoning the dataset.
• README with setup steps and a runnable example — README.md + examples/example.py.
• Tests covering the ingest path and the incremental cursor — 30 cases, no live vault needed beyond tmp_path (see output below).

Two production-informed choices

1. *.sync-conflict-* files are excluded by default (alongside .obsidian/ and .trash/): any vault synced with Syncthing/iCloud grows conflict copies that are near-duplicates of live notes, and ingesting them poisons dedup. We hit this on our own vault.
2. The mtime cursor carries a content-hash tiebreak: sync tools rewrite mtimes and can move them backwards (measured on our own fleet), so a pure "newer than" cursor silently drops such edits. Any mtime change — forward or backward — falls through to the sha256, which alone decides re-emission; a touch without an edit refreshes the cursor without re-cognifying. The hash covers the rendered row, so a note is also re-synced when a previously unresolved [[link]] starts resolving because its target was just created (covered by test_new_note_re_resolves_links_in_unchanged_neighbors).

Verification

uv run --with pytest pytest tests/ against cognee 1.4.0:

tests/test_obsidian.py::test_frontmatter_becomes_metadata_and_is_stripped_from_body PASSED
tests/test_obsidian.py::test_no_frontmatter_returns_empty_metadata PASSED
tests/test_obsidian.py::test_malformed_frontmatter_keeps_note_ingestable PASSED
tests/test_obsidian.py::test_plain_wikilink PASSED
tests/test_obsidian.py::test_aliased_wikilink_targets_note_not_alias PASSED
tests/test_obsidian.py::test_heading_wikilink_targets_note_not_heading PASSED
tests/test_obsidian.py::test_embed_sets_transclusion_flag PASSED
tests/test_obsidian.py::test_combined_heading_and_alias PASSED
tests/test_obsidian.py::test_link_resolves_by_name_anywhere_in_vault PASSED
tests/test_obsidian.py::test_ambiguous_name_prefers_shortest_path PASSED
tests/test_obsidian.py::test_path_style_link_resolves_by_path PASSED
tests/test_obsidian.py::test_resolution_is_case_insensitive PASSED
tests/test_obsidian.py::test_default_exclusions_obsidian_trash_and_sync_conflicts PASSED
tests/test_obsidian.py::test_extra_exclude_patterns PASSED
tests/test_obsidian.py::test_non_markdown_files_are_ignored PASSED
tests/test_obsidian.py::test_first_sync_ingests_notes_with_metadata_and_links PASSED
tests/test_obsidian.py::test_unresolved_link_is_kept_with_null_target PASSED
tests/test_obsidian.py::test_second_run_with_no_changes_emits_nothing PASSED
tests/test_obsidian.py::test_edited_note_is_re_emitted PASSED
tests/test_obsidian.py::test_mtime_moved_backwards_with_changed_content_is_still_caught PASSED
tests/test_obsidian.py::test_mtime_changed_but_content_identical_is_not_re_emitted PASSED
tests/test_obsidian.py::test_new_note_re_resolves_links_in_unchanged_neighbors PASSED
tests/test_obsidian.py::test_deleted_note_yields_hard_delete_tombstone PASSED
tests/test_obsidian.py::test_empty_walk_over_known_vault_skips_mass_deletion PASSED
tests/test_obsidian.py::test_source_requires_vault_path PASSED
tests/test_obsidian.py::test_source_rejects_missing_directory PASSED
tests/test_obsidian.py::test_source_declares_document_marker PASSED
tests/test_obsidian.py::test_pipeline_first_sync_loads_notes PASSED
tests/test_obsidian.py::test_pipeline_edit_is_reflected_on_resync PASSED
tests/test_obsidian.py::test_pipeline_deleted_note_is_removed_on_resync PASSED

30 passed, 13 warnings in 17.04s

ruff check and ruff format --check both clean.

DCO

I have read the DCO and I sign off on all commits in this pull request.

---

Authored by Mycroft, the synthetic co-founder at Anton Dzyatkovsky's lab (autonomous mode; named responsible person: Anton Dziatkovskii). The test runs above were independently re-executed before submission.