[V2] Add TaskAction garbage collector for terminal CRDs

[pingsutw]

Problem

Terminal TaskAction CRDs (Succeeded/Failed) remain in the cluster indefinitely after completion. This wastes etcd storage and slows down list operations, especially as the number of completed tasks grows over time.

Proposed Solution

Add a label-based TTL garbage collector for terminal TaskActions, modeled after the propeller FlyteWorkflow GC (flytepropeller/pkg/controller/garbage_collector.go).

Design

1. Terminal labeling: When a TaskAction reaches a terminal state (Succeeded/Failed), stamp it with flyte.org/termination-status=terminated and flyte.org/completed-time=<UTC hour> labels
2. Background GC loop: A manager.Runnable that periodically lists terminated TaskActions, filters by completed-time label (lexicographically ordered), and deletes expired ones
3. Configuration: GCConfig with Interval (how often GC runs) and MaxTTL (time-to-live for terminal TaskActions)

Key Design Decisions

• List + filter + delete (not DeleteAllOf): K8s label selectors don't support "less than" on string values, so we list all terminated TaskActions and filter client-side by hour label
• Separate metadata update: Labels require r.Update() (not r.Status().Update()). One extra API call per terminal transition, but only once per TaskAction lifetime
• Upgrade path: Pre-existing terminal TaskActions get labels on next reconcile via the terminal short-circuit path

Implementation

PR: #6994

[runllm]

It seems likely that your proposed solution closely mirrors the FlyteWorkflow garbage collector logic in flytepropeller/pkg/controller/garbage_collector.go, which deletes completed workflows older than a configured TTL by listing, filtering, and deleting them in a background loop. The approach of labeling terminal CRDs and running a periodic GC loop is consistent with this pattern, and the need to list and filter client-side due to K8s label selector limitations is also reflected in the FlyteWorkflow GC implementation. The use of a configuration struct for interval and TTL is also present in the referenced code (garbage_collector.go).

However, there is no explicit mention of a TaskAction garbage collector or the exact label keys (flyte.org/termination-status, flyte.org/completed-time) in the provided data sources. The design and rationale you describe are in line with the FlyteWorkflow GC, but the specific implementation for TaskAction CRDs is not present in the referenced code. Would you like a step-by-step breakdown of how to adapt the FlyteWorkflow GC pattern for TaskAction CRDs?

Sources:

• garbage_collector.go

---

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @RunLLM to ask follow-up questions. See runllm.com for more!