Skip to content

fix(trail): record_count/summary/export include unflushed buffered records (#150) - #164

Merged
rajfirke merged 1 commit into
mainfrom
fix/150-buffered-reads
Sep 3, 2026
Merged

fix(trail): record_count/summary/export include unflushed buffered records (#150)#164
rajfirke merged 1 commit into
mainfrom
fix/150-buffered-reads

Conversation

@rajfirke

@rajfirke rajfirke commented Sep 3, 2026

Copy link
Copy Markdown
Owner

What does this PR do?

In buffered mode, record_count, summary(), and export() all queried only the backend, giving a stale view of the trail whenever records were sitting in the write buffer:

  • record_count returned 0 (or the flushed count) until the next background flush
  • summary() reported stale provenance/freshness/source aggregates — a MISSING provenance in the buffer was invisible
  • export() silently omitted pending writes entirely

Fix:

WriteBuffer gains two helpers:

  • __len__ — delegates to pending so callers can use len(buffer)
  • pending_records — returns a thread-safe snapshot of the buffer contents

ContextTrail changes:

  • record_count adds self._buffer.pending to the backend count without flushing (read-only)
  • summary() appends self._buffer.pending_records to the backend snapshot before aggregating
  • export() calls self._buffer.flush() before reading — the export must be complete; a non-flushing approach would return an inconsistent snapshot here

Also cleans up two leftover issues from the [1.1.0] offset addition:

  • Removes the stray # Added offset parameter to truly handle datasets of any size without hard caps comment in trail.py
  • Renames test_context_trail_query_supports_offsettest_paginated_mock_trail_returns_distinct_pages to reflect that it tests PaginatedMockTrail, not ContextTrail.query()

Checklist

  • Tests added/updated for the change
  • ruff check src/ tests/ passes
  • ruff format --check src/ tests/ passes
  • mypy src/provena/ passes
  • pytest passes with no failures
  • CHANGELOG.md updated (if user-facing change)

Related Issues

Closes #150


Note: PR #161 covered the same fix. After a week without the requested revisions, this replaces it with the corrected implementation. Closing #161 with a note.

…cords (#150)

In buffered mode, all three read paths queried only the backend, so:
- record_count was 0 until the next background flush
- summary() reported stale provenance/freshness aggregates
- export() silently omitted pending writes

WriteBuffer gains __len__ (delegates to pending) and pending_records (thread-safe
snapshot). record_count adds the pending count without flushing; summary() appends
the snapshot to the backend records; export() flushes explicitly so its backend
read is always complete.

Also removes the stray # Added offset parameter comment left over from the [1.1.0]
offset addition, and renames test_context_trail_query_supports_offset to reflect
that it tests PaginatedMockTrail, not ContextTrail.query().

Closes #150
@rajfirke rajfirke mentioned this pull request Sep 3, 2026
@rajfirke
rajfirke merged commit 4182dcf into main Sep 3, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

summary(), export(), and record_count exclude unflushed buffered records — stale view in buffered mode

1 participant