Skip to content

[Roadmap] Preset-level threshold governance and drift alerts #2

Description

@ZoeZYZY

Goal

Operationalize preset-level quality control using run telemetry.

Current status

  • by_preset thresholds exist
  • dashboard supports by-preset stats

Proposed scope

  • Add weekly trend report for failure/retry by preset
  • Add threshold drift detection rule (alert when preset failure_rate jumps > X)
  • Define rollback strategy for unstable presets

Success criteria

  • Preset regressions are visible within one release cycle
  • Quality incidents can be traced to preset + provider quickly

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestmonitoringTelemetry and dashboardqualityGeneration quality, evaluation, thresholds

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions