Skip to content

Add rose_prune operator support for pruning data older than a specified cycle #2939

@cpelley

Description

@cpelley

Currently, workflows we run periodically can accumulate data that escapes housekeeping -- particularly in scenarios where cycles are skipped to catch up with real-time. The workaround of defining multiple fixed offsets to reattempt housekeeping in later cycles is neither robust nor intuitive. It also lacks the clarity and intent expressed by simply stating that data older than a given cycle should be pruned.

We propose enhancing rose_prune with operator support (e.g. <=, <) to allow pruning based on relative cycle age. This would align the tool’s behaviour more closely with user intent and improve robustness in real-world operational contexts.

Motivation:

  • Simplifies configuration by replacing long lists of offsets with a single, expressive rule.
  • Reduces risk of missed housekeeping due to skipped cycles.
  • Encourages centralised, consistent use of rose_prune rather than bespoke project scripts.

Example:

prune{share/data/dummy/}=-PT1H -PT2H -PT3H -PT4H -PT5H -PT6H -PT7H -PT8H -PT9H -PT10H -PT11H -PT12H -PT24H -PT25H -PT26H -PT27H -PT28H -PT29H

Proposed syntax:

prune{share/data/dummy/}='<=-PT1H'

Additional Notes:

  • This feature would be particularly useful for suites with high-frequency cycling and long runtimes.
  • For several years, we’ve handled this logic outside of rose_prune due to its limitations. Bringing it into the core tool would streamline operations and reduce duplication.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions