-
Notifications
You must be signed in to change notification settings - Fork 20
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Summary
While updating parameter handling code, I noticed that the database records a job as being archived before all cleanup is complete. Example:
https://github.com/statgen/cloudgene/blob/950b8b8bd5865482e0114f19f70d17fa83a753bf/src/main/java/cloudgene/mapred/api/v2/admin/ArchiveJob.java#L90-L96
In theory, in the event of a temporary communication failure with external workspace, this could lead to files being retained even after the official records say a file was deleted. This would manifest as significantly higher storage usage in the external workspace. Depending on which files were not deleted, it could also have other implications.
Suggested actions
- Change the order of instructions so that the DB archive state is only set after all cleanup operations are complete.
- Audit other delete/archive operations for similar out-of-order behaviors.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working