Description
Motivation: Why do you think this is important?
As suggested in #2633, cache eviction/cleanup functionality provides Flyte users as well as administrators with more control about what data is to be cached by Flyte and allows for cleanup of no longer required results.
Goal: What should the final outcome look like, ideally?
This issue tracks implementation of the first suggested part of the RFC, providing a cache eviction override for a single execution.
flyteadmin and flytepropeller should allow users to skip any existing cache for a single execution, overwriting any stored data in datacatalog after the execution finishes successfully.
All relevant Flyte clients (flyteconsole, flytekit, flytectl) should support the new override functionality.
Describe alternatives you've considered
As no similar functionality exists (except manually deleting entries in datacatalog's Postgres database and blob storage), no alternatives were considered in the RFC.
Propose: Link/Inline OR Additional context
https://hackmd.io/qOztkaj4Rb6ypodvGEowAg?view#Cache-eviction-override-for-a-single-execution
Are you sure this issue hasn't been raised already?
- Yes
Have you read the Code of Conduct?
- Yes