IntentLog - #850
Draft
lemaitre-aneo wants to merge 15 commits into
Draft
Conversation
lemaitre-aneo
force-pushed
the
fl/intent-log
branch
6 times, most recently
from
February 23, 2025 17:31
55df31b to
111dd88
Compare
lemaitre-aneo
force-pushed
the
fl/intent-log
branch
3 times, most recently
from
March 4, 2025 21:34
a8c54f3 to
901e8f4
Compare
lemaitre-aneo
force-pushed
the
fl/intent-log
branch
from
March 5, 2025 09:24
901e8f4 to
5625b41
Compare
aneojgurhem
added a commit
that referenced
this pull request
Sep 19, 2025
# Motivation We observed cases of tasks that were not properly submitted due to interruptions or errors during submission. This PR aims to make submission related RPCs atomic so that, when there is an issue, no tasks or results are left in an inconsistent state. # Description A try/catch block was added around critical parts of the code where the tasks/resutls are deleted from the database/object storage upon exception. # Testing Unit tests were implemented to check that tasks and results are properly deleted. # Impact - Tasks that are in Pending state due to errors during submission should be deleted and not appear in monitoring anymore. - Issues during result creation should not leave partial results created # Additional Information - Rollback after task submission by another task is not properly implemented because owner id in the result is changed without keeping the old value. It requires a change in the data scheme to keep the old value and put it back on if needed. This case should be treated at some point as it is needed to properly rollback sub tasks when task is reacquired after crash during postprocessing. - This correction is not completely foolproof as it assumes the service to be able to execute the catch clause. It may not be executed in case of sudden crash of the service. Moreover, connection loss to the data plane may not allow to run the code within the catch properly. More complete solutions depends on #850
italo1aguiar-cacib
pushed a commit
to CACIB-ArmoniK-Admin/ArmoniK.Core
that referenced
this pull request
Mar 2, 2026
We observed cases of tasks that were not properly submitted due to interruptions or errors during submission. This PR aims to make submission related RPCs atomic so that, when there is an issue, no tasks or results are left in an inconsistent state. A try/catch block was added around critical parts of the code where the tasks/resutls are deleted from the database/object storage upon exception. Unit tests were implemented to check that tasks and results are properly deleted. - Tasks that are in Pending state due to errors during submission should be deleted and not appear in monitoring anymore. - Issues during result creation should not leave partial results created - Rollback after task submission by another task is not properly implemented because owner id in the result is changed without keeping the old value. It requires a change in the data scheme to keep the old value and put it back on if needed. This case should be treated at some point as it is needed to properly rollback sub tasks when task is reacquired after crash during postprocessing. - This correction is not completely foolproof as it assumes the service to be able to execute the catch clause. It may not be executed in case of sudden crash of the service. Moreover, connection loss to the data plane may not allow to run the code within the catch properly. More complete solutions depends on aneoconsulting#850
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
[Include the reason behind these changes and any relevant context.]
Description
[Provide a detailled explanation of the modifications you have made. Link any related issues.]
Testing
[When applicable, detail the testing you have performed to ensure that these changes function as intended. Include information about any added tests.]
Impact
[Discuss the impact of your modifications on ArmoniK. This might include effects on performance, configuration, documentation, new dependencies, or changes in behaviour.]
Additional Information
[Any additional information that reviewers should be aware of.]
Checklist