vine: garbage collect at workers #3347
Unanswered
btovar
asked this question in
TaskVine Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
When an application detects that a file is not needed anymore it can call
vine_remove_fileand this deletes the file from the workers, and from the manager's file table. When not using temp files this seems like the correct behavior.However, when using temp files, the application may decide to retrieve some of the files as checkpoints of the computation. Given space considerations, it may be desirable to delete this files from the workers but not entirely from the manager. Should a worker be lost with its temp files, the manager can send the saved files so that the computation does not have to start from scratch.
Beta Was this translation helpful? Give feedback.
All reactions