Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return emitted data in case of emission error. #262

Merged
merged 21 commits into from
Feb 13, 2024
Merged

Return emitted data in case of emission error. #262

merged 21 commits into from
Feb 13, 2024

Conversation

Totodore
Copy link
Owner

@Totodore Totodore commented Feb 6, 2024

Fix #172

There are two remaining questions :

  • What should be done with binary packets. Because a message can contain N binary payloads it means that we would need N+1 Permits for each emit call.
  • What should be done when broadcasting. What is the performance cost of asking X Permits from N sockets at the same time ? Because packet is encoded only once and then cloned many times to be sent. Maybe it would be better to just clone the data before sending anything so it is possible to send it back to the user.
    Also broadcasting is done through Operators, and Operators are not always used to broadcast, they can be used to simply add a timeout or a binary payload when emitting. So the emit signature needs to be the same from the Operator fn even if it is not broadcasting.
  • The new reserve_many API on tokio v1.36 resolves the first question.
  • The second problem still needs to be resolved.

@Totodore
Copy link
Owner Author

What should be done when broadcasting. What is the performance cost of asking X Permits from N sockets at the same time ? Because packet is encoded only once and then cloned many times to be sent. Maybe it would be better to just clone the data before sending anything so it is possible to send it back to the user.
Also broadcasting is done through Operators, and Operators are not always used to broadcast, they can be used to simply add a timeout or a binary payload when emitting. So the emit signature needs to be the same from the Operator fn even if it is not broadcasting.

Because operators are now splitted between ConfOperators and BroadcastOperators it is possible to separate the code between broadcasting and single emission with configuration (binary, timeout, etc). There are two main advantages:

  • Single emission with configuration is more performant because it is directly done on the socket. Before the adapter was called to fetch the concerned socket.
  • It is possible to return the emitted data in case of emission error for:
    • emit
    • emit_with_ack

@Totodore
Copy link
Owner Author

Maybe it would be worth it to think about a better pattern, in order to avoid code duplication between ConfOperators and BroadcastOperators...

@Totodore Totodore enabled auto-merge (squash) February 13, 2024 22:54
@Totodore Totodore merged commit 0e0fe75 into main Feb 13, 2024
12 checks passed
@Totodore Totodore deleted the ft-try_emit branch March 9, 2024 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rework of errors when emitting data
1 participant