Found during review of #2215.
Summary
On a targeted/identity reimport with skip_if_unchanged=True, a document that is byte-identical to what is stored records a PendingDocumentAnnotations row with status=DONE, id_map={} (opencontractserver/tasks/import_tasks_v2.py:844-858). The document therefore contributes nothing to the run's aggregated id-map.
finalize_corpus_import_relationships (opencontractserver/tasks/doc_tasks.py:1964+) aggregates only DONE rows' id_maps and passes the result to _import_v2_relationships (import_tasks_v2.py:1373-1428), which silently drops any relationship endpoint whose export-local id is not in that map (lines 1403-1412) — no log, no warning.
Net effect
On a pack refresh where document A changed and document B (referenced by a corpus-level relationship) did not, the relationship silently loses its B endpoint with zero signal.
skip_if_unchanged is the intended common case for a re-shipped authority pack, so this is not a rare edge case — it should progressively erode corpus-level relationships across repeated pack refreshes.
Why it was not fixed in #2215
The fix is not a guard. To keep the relationship alive you must reconstruct the export-local-id → existing-annotation-id mapping for a document you deliberately did not re-parse. That means either:
- persisting the prior run's id-map so a converged document can replay it, or
- re-deriving the mapping from stored annotations at reimport time.
Either way it introduces new persisted data or a new derivation path and changes _import_document_with_annotations' contract with finalize_corpus_import_relationships — an architectural change rather than a local correction.
Minimum interim improvement
Even without the full fix, _import_v2_relationships should log dropped endpoints instead of discarding them silently, so the erosion is observable.
Test gap
No test covers a corpus-level relationship spanning one changed and one unchanged document across an incremental reimport.
Found during review of #2215.
Summary
On a targeted/identity reimport with
skip_if_unchanged=True, a document that is byte-identical to what is stored records aPendingDocumentAnnotationsrow withstatus=DONE, id_map={}(opencontractserver/tasks/import_tasks_v2.py:844-858). The document therefore contributes nothing to the run's aggregated id-map.finalize_corpus_import_relationships(opencontractserver/tasks/doc_tasks.py:1964+) aggregates onlyDONErows'id_maps and passes the result to_import_v2_relationships(import_tasks_v2.py:1373-1428), which silently drops any relationship endpoint whose export-local id is not in that map (lines 1403-1412) — no log, no warning.Net effect
On a pack refresh where document A changed and document B (referenced by a corpus-level relationship) did not, the relationship silently loses its B endpoint with zero signal.
skip_if_unchangedis the intended common case for a re-shipped authority pack, so this is not a rare edge case — it should progressively erode corpus-level relationships across repeated pack refreshes.Why it was not fixed in #2215
The fix is not a guard. To keep the relationship alive you must reconstruct the export-local-id → existing-annotation-id mapping for a document you deliberately did not re-parse. That means either:
Either way it introduces new persisted data or a new derivation path and changes
_import_document_with_annotations' contract withfinalize_corpus_import_relationships— an architectural change rather than a local correction.Minimum interim improvement
Even without the full fix,
_import_v2_relationshipsshould log dropped endpoints instead of discarding them silently, so the erosion is observable.Test gap
No test covers a corpus-level relationship spanning one changed and one unchanged document across an incremental reimport.