Once sufficient run history exists in the PostgreSQL database (#162), implement
a pattern detection layer that watches for anomalies across runs rather than
within a single run.
Examples of patterns worth detecting:
- a specific stage failing repeatedly across runs
- a specific sample consistently missing R2
- file size outliers occurring at a higher rate than baseline
- NIRD quota consumption rate trending toward exhaustion
- delivery times increasing over time (infrastructure degradation signal)
- runs from a specific instrument serial number producing worse QC metrics
Implementation notes:
Depends on: #162 (PostgreSQL), #145 (Teams notifications), #200 (file size
statistics as input signal)
Related: #139 (detect when a single run has not completed in due time).
#139 is per-run timeout detection; this ticket is cross-run pattern analysis.
Both feed into the same anomaly alerting layer.
Not in scope: machine learning, statistical modelling. Simple threshold and
trend detection only for v1.
Once sufficient run history exists in the PostgreSQL database (#162), implement
a pattern detection layer that watches for anomalies across runs rather than
within a single run.
Examples of patterns worth detecting:
Implementation notes:
at least 3-6 months of data exists in feature database: create a postgresql instance that will contain 2 databases: demux and shinylims #162
no side effects
Depends on: #162 (PostgreSQL), #145 (Teams notifications), #200 (file size
statistics as input signal)
Related: #139 (detect when a single run has not completed in due time).
#139 is per-run timeout detection; this ticket is cross-run pattern analysis.
Both feed into the same anomaly alerting layer.
Not in scope: machine learning, statistical modelling. Simple threshold and
trend detection only for v1.