Skip to content

feature: cross-run pattern detection and anomaly alerting #204

Description

@georgemarselis-nvi

Once sufficient run history exists in the PostgreSQL database (#162), implement
a pattern detection layer that watches for anomalies across runs rather than
within a single run.

Examples of patterns worth detecting:

  • a specific stage failing repeatedly across runs
  • a specific sample consistently missing R2
  • file size outliers occurring at a higher rate than baseline
  • NIRD quota consumption rate trending toward exhaustion
  • delivery times increasing over time (infrastructure degradation signal)
  • runs from a specific instrument serial number producing worse QC metrics

Implementation notes:

Depends on: #162 (PostgreSQL), #145 (Teams notifications), #200 (file size
statistics as input signal)

Related: #139 (detect when a single run has not completed in due time).
#139 is per-run timeout detection; this ticket is cross-run pattern analysis.
Both feed into the same anomaly alerting layer.

Not in scope: machine learning, statistical modelling. Simple threshold and
trend detection only for v1.

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions