Skip to content

[Core feature] Cache eviction override for a single execution #2867

Open
flyteorg/flyteconsole
#851
@MorpheusXAUT

Description

@MorpheusXAUT

Motivation: Why do you think this is important?

As suggested in #2633, cache eviction/cleanup functionality provides Flyte users as well as administrators with more control about what data is to be cached by Flyte and allows for cleanup of no longer required results.

Goal: What should the final outcome look like, ideally?

This issue tracks implementation of the first suggested part of the RFC, providing a cache eviction override for a single execution.

flyteadmin and flytepropeller should allow users to skip any existing cache for a single execution, overwriting any stored data in datacatalog after the execution finishes successfully.

All relevant Flyte clients (flyteconsole, flytekit, flytectl) should support the new override functionality.

Describe alternatives you've considered

As no similar functionality exists (except manually deleting entries in datacatalog's Postgres database and blob storage), no alternatives were considered in the RFC.

Propose: Link/Inline OR Additional context

https://hackmd.io/qOztkaj4Rb6ypodvGEowAg?view#Cache-eviction-override-for-a-single-execution

Are you sure this issue hasn't been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions