Couple external-directory Lucene index commits to GigaMap.store() (#715)

[fh-ms]

Problem

Fixes #715.

The LuceneIndex for GigaMap committed eagerly at mutation time — every internalAdd / internalAddAll / internalRemove / internalRemoveAll / internalUpdateIndices flushed and committed the IndexWriter immediately (because LuceneContext.autoCommit() defaulted to true). The actual consistency guarantee then silently depended on the directory type:

Directory	autoCommit=true (default)	autoCommit=false
GraphDirectory (embedded, default — index lives in the persisted fileEntries map)	commit targets the in-memory map; reaches disk only on store(). Consistent at store boundaries.	broken: no commit means the writer may not flush into fileEntries, so store() could persist an empty/stale index.
MMap(path) (external, on-disk index)	commit hits disk at mutation time, decoupled from store(). A mutation with no following store() (or a crash in between) left the on-disk index diverged from the persisted entities.	no commit at all unless commit() was called manually.

This contradicts the "save it when I tell you to save" expectation (discussion #708).
On top of that, autoCommit=false was effectively unreachable for external users:
LuceneContext.Default's constructor is package-private and no factory exposed the flag.

Changes

• LuceneContext — added a public factory overload
  New(directoryCreator, analyzerCreator, documentPopulator, autoCommit).
  The existing four factories still default to autoCommit=true, so behavior is unchanged for current users. The flag is stored inverted as manualCommit:
  a field added to a persisted type defaults to false when an older store is loaded, and storing manualCommit makes that default mean autoCommit()==true, preserving the legacy behavior on reload with no migration.
• LuceneIndex.Default.internalCommitOnStore() — flushes + commits the writer when autoCommit is disabled and there are uncommitted changes (no-op otherwise).
• BinaryHandlerLuceneIndexDefault — overrides store(...) to call internalCommitOnStore() before serialization. The ordering matters: super.store
  stores the fileEntries reference and then its contents, so the commit must populate fileEntries first. This couples the commit to the GigaMap.store() boundary for both directory types and fixes the latent GraphDirectory stale-persist bug.
• Docs — documented the autoCommit × directory-type matrix on autoCommit() and the remaining external-directory caveat: the index and the GigaMap storage are separate persistence targets, so coupling aligns the commits temporally but is not a single atomic transaction across both (a crash mid-store() can still diverge). The embedded graph directory has no such gap.

Behavior

• Default (autoCommit=true) is unchanged.
• With autoCommit=false, the writer is flushed+committed exactly once, at the GigaMap.store() boundary (or via an explicit commit() between stores).

Testing

• Full gigamap-lucene suite green (61 tests).
• New LuceneStoreBoundaryCommitTest:
  • MMap: a read-only reader over the on-disk commit shows the doc count tracking store() calls, not mutations — proving store-boundary commit and no eager commit.
  • GraphDirectory: index data survives persist + reload with autoCommit=false.
• Confirmed both new tests fail without the fix (graph reload returns 0 docs; MMap finds nothing committed on disk), then pass once the commit hook is restored.

[Copilot]

Pull request overview

Couples Lucene index commit behavior to the GigaMap.store() boundary (when autoCommit=false) so that external-directory indexes don’t durably diverge from persisted entities due to eager mutation-time commits, while preserving legacy behavior (autoCommit=true) for existing users.

Changes:

• Adds a public LuceneContext.New(..., autoCommit) factory overload and persists the setting via an inverted manualCommit flag to maintain backward-compatible defaults on reload.
• Introduces a store-boundary commit hook (internalCommitOnStore) and invokes it from BinaryHandlerLuceneIndexDefault.store(...) before serialization.
• Adds a new test covering store-boundary commit semantics for both MMap (external) and GraphDirectory (embedded) modes.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
gigamap/lucene/src/main/java/org/eclipse/store/gigamap/lucene/LuceneContext.java	Exposes autoCommit configuration via a new factory overload and documents directory-type semantics.
gigamap/lucene/src/main/java/org/eclipse/store/gigamap/lucene/LuceneIndex.java	Adds store-boundary commit behavior when autoCommit=false and updates commit API docs accordingly.
gigamap/lucene/src/main/java/org/eclipse/store/gigamap/lucene/BinaryHandlerLuceneIndexDefault.java	Hooks the store-boundary commit into persistence by committing before LuceneIndex serialization.
gigamap/lucene/src/test/java/org/eclipse/store/gigamap/lucene/LuceneStoreBoundaryCommitTest.java	Verifies external and embedded directory behavior matches store-boundary commit expectations.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.