Skip to content

chore(perf): Avoid message cloning in Sink#3203

Merged
BulkBeing merged 5 commits into
mainfrom
avoid-cloning-sink
Feb 6, 2026
Merged

chore(perf): Avoid message cloning in Sink#3203
BulkBeing merged 5 commits into
mainfrom
avoid-cloning-sink

Conversation

@BulkBeing
Copy link
Copy Markdown
Contributor

@BulkBeing BulkBeing commented Feb 6, 2026

What this PR does / why we need it

Avoids cloning the messages in Sink wherever possible.

Testing

Tested manually by sending 100,000 messages and validating all messages on the Sink side.

Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 6, 2026

Codecov Report

❌ Patch coverage is 92.85714% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 80.24%. Comparing base (7f12e5e) to head (87d9429).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
rust/numaflow-core/src/sinker/actor.rs 92.85% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3203      +/-   ##
==========================================
- Coverage   80.26%   80.24%   -0.02%     
==========================================
  Files         297      297              
  Lines       67639    67640       +1     
==========================================
- Hits        54289    54281       -8     
- Misses      12802    12811       +9     
  Partials      548      548              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
@BulkBeing BulkBeing marked this pull request as ready for review February 6, 2026 04:23
@vigith vigith changed the title chore: Avoid message cloning in Sink chore(perf): Avoid message cloning in Sink Feb 6, 2026
@vigith vigith added the performance Performance Improvements label Feb 6, 2026
Comment thread rust/numaflow-core/src/sinker/actor.rs Outdated
Comment on lines +103 to +104
// Don't increment i here, since we swapped the element at i with the last element
messages_to_retry.swap_remove(i);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

won't we lose ordering because of replace with the last element during unhappy path?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.. Should we preserve the order ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, because during retries, we do not want to switch semantics to unordered.

Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
@BulkBeing BulkBeing requested a review from vigith February 6, 2026 05:20
@BulkBeing BulkBeing merged commit 498175d into main Feb 6, 2026
27 of 28 checks passed
@BulkBeing BulkBeing deleted the avoid-cloning-sink branch February 6, 2026 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Performance Improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants