Skip to content

fix: send explicit eot from stream receiver task to stream execute task#3418

Draft
vaibhavtiwari33 wants to merge 3 commits intomainfrom
map-stream-loss
Draft

fix: send explicit eot from stream receiver task to stream execute task#3418
vaibhavtiwari33 wants to merge 3 commits intomainfrom
map-stream-loss

Conversation

@vaibhavtiwari33
Copy link
Copy Markdown
Contributor

@vaibhavtiwari33 vaibhavtiwari33 commented May 8, 2026

What this PR does / why we need it

Whenever the stream receiver task, a single task responsible for receiving responses from udf server and streaming them back to the respective execute tasks, drops the sender used for streaming back responses to execute tasks; the execute tasks see that as a signal that all the responses have been sent by the receiver task.

We should send an explicit EOT from from the receiver task to let the execute task know that we're done sending responses.

Related issues

Fixes #3417

Testing

  • Updated the unit tests
  • Testing using the validation platform

Special notes for reviewers

Anything notable for review (risk, rollout, follow-ups).

Signed-off-by: Vaibhav Tiwari <vaibhav.tiwari33@gmail.com>
Signed-off-by: Vaibhav Tiwari <vaibhav.tiwari33@gmail.com>
@vaibhavtiwari33 vaibhavtiwari33 marked this pull request as ready for review May 8, 2026 13:45
@codecov
Copy link
Copy Markdown

codecov Bot commented May 8, 2026

Codecov Report

❌ Patch coverage is 85.71429% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.68%. Comparing base (e07240b) to head (df455a0).

Files with missing lines Patch % Lines
rust/numaflow-core/src/mapper/map/stream.rs 85.71% 12 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3418      +/-   ##
==========================================
+ Coverage   82.64%   82.68%   +0.03%     
==========================================
  Files         307      307              
  Lines       76051    76089      +38     
==========================================
+ Hits        62853    62911      +58     
+ Misses      12645    12620      -25     
- Partials      553      558       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vaibhavtiwari33 vaibhavtiwari33 self-assigned this May 8, 2026
@vaibhavtiwari33 vaibhavtiwari33 added bug Something isn't working area/udf User defined functions dataplane Rust Implementation of Data Plane labels May 8, 2026
@vaibhavtiwari33 vaibhavtiwari33 marked this pull request as draft May 9, 2026 01:03
…n cancellation token in execute loop

Signed-off-by: Vaibhav Tiwari <vaibhav.tiwari33@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/udf User defined functions bug Something isn't working dataplane Rust Implementation of Data Plane

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Map stream can ack and return partial results from udf server

1 participant