Add true time based flush behaviour to Coalesce transform#1997
Conversation
Merging this PR will not alter performance
Comparing Footnotes
|
There was a problem hiding this comment.
Pull request overview
This PR adds true time-based flushing to the Coalesce transform by introducing a per-connection background timer that can wake the chain to flush buffered messages, rather than only flushing opportunistically when new messages arrive.
Changes:
- Implement a per-connection Tokio task in
Coalescethat periodically signalsforce_run_chainand triggers a flush when the buffer is non-empty. - Update config validation/error messaging around flush settings (including requiring millis > 0 to enable the timer).
- Refresh tests and documentation to reflect the new timer-driven behavior and semantics.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| shotover/src/transforms/coalesce.rs | Adds timer-driven flush behavior, new validation logic, and expanded tests for timer/count interactions. |
| shotover/src/config/topology.rs | Updates topology validation test expectations to match new Coalesce validation messaging. |
| docs/src/transforms.md | Documents the new per-connection timer behavior and clarifies configuration semantics. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
The additional comments here look good and clear. Have approved the PR |
Closes : #1673
This PR introduces a background timer that will force the Transfrom chain to run if a time based flush definition is present when the time since last flush is greater than the time defined.
Last flush is considered to be the time