fix: consolidate streaming, Arrow, and event contracts#633
Merged
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #633 +/- ##
==========================================
- Coverage 75.79% 75.77% -0.03%
==========================================
Files 473 473
Lines 65121 65170 +49
Branches 9713 9723 +10
==========================================
+ Hits 49361 49383 +22
- Misses 12518 12530 +12
- Partials 3242 3257 +15
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
cofin
marked this pull request as draft
July 10, 2026 00:24
cofin
changed the title
cofin
marked this pull request as ready for review
July 10, 2026 04:58
cofin
added a commit
that referenced
this pull request
Jul 10, 2026
## Summary
- dedicate one long-lived listener resource per native backend instance
while publishers use short pooled sessions
- add typed sync/async `publish_many()` with ordered IDs, explicit
grouped transactions, and an ordered fallback
- bulk-insert durable rows and emit one compact `{marker_id,
batch_size}` wakeup marker per channel atomically
- recover missing markers through independent `event_poll_interval`
reconciliation and immediately drain every recovered row
- cancel async listener waits concurrently, bound sync joins, and
prevent table polling faster than the configured interval
- preserve durable batch delivery order and add
`SQLSpecChannelsBackend.publish_many(data, channels)` for consumer
integration
- document canonical transport semantics, adapter support, connection
ownership, and recovery behavior
## Integration
This PR now targets `main` directly after #633 and #634 merged. Its
event-only commits are rebased onto merge commit `ff0b8cb51`, so the
final branch validates the consolidated C4 contracts and lazy-typing
implementation together.
## Correctness notes
Durable rows are the source of truth. Native markers are wakeup hints,
not batch envelopes. Row insertion and marker publication share one
explicit transaction, marker publication failure rolls back the rows,
duplicate markers are suppressed, and one reconciliation timeout enters
drain mode until the queue is empty. Durable transports remain
competing-consumer queues rather than browser fan-out.
JSON payloads and metadata remain native values through batch parameter
preparation so adapter-specific encoders receive the same input shape as
single-event publication.
## Validation
- listener lifecycle and native backend factory tests
- 1,000-event batch, ordering, transaction rollback, autocommit
restoration, session/statement count, empty/fallback, marker
loss/duplication, polling, and metadata tests
- real asyncpg, psqlpy, and psycopg PostgreSQL integration coverage for
single-plus-batch IDs, delivery order, payloads, and metadata
- SQLite durable order and Litestar channel integrations
- focused event suite, Ruff, mypy, pyright, docs, and docs linkcheck
- full GitHub CI is running for the final post-merge rebased commit
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
notify,notify_queue,poll_queue,aq, andtxeventqReview fixes
TypeErroris not retriedclose()still clean up correctlyevents_backendfeatures are honored when extension configuration does not override themStack
This is the base PR for #635, which contains listener lifecycle, batching, polling/recovery, and final transport documentation.
Validation
All GitHub checks are green.