Update AutoGrowth upload pipeline for high-frequency OD pre-aggregation and flexible calibration inputs

[Copilot]

This update extends the upload flow to better handle real-world acquisition and calibration formats: high-frequency PioReactor streams can now be pre-aggregated before processing, and calibration uploads now accept both delimiter variants and Excel files. It also clarifies ECOD sensitivity behavior directly in the UI.

• Raw OD pre-aggregation for sub-15s sampling

  • Added an optional upload-page control (PioReactor-only) to aggregate raw OD measurements when median sampling interval is below 15 seconds.
  • Implemented pre-processing aggregation in process_od_pioreactor(...) via maybe_aggregate_high_frequency_raw_data(...), with summary messaging on whether aggregation was applied.

• ECOD sensitivity UX clarification

  • Updated ECOD slider help text to explicitly state: lower factor = more sensitive, higher factor = less sensitive.

• Calibration ingestion: delimiter + Excel support

  • Introduced read_od_adjustment_table(...) to centralize calibration parsing.
  • CSV/TXT parsing now supports both , and ; separators (sniffed from file header).
  • Added .xlsx / .xls support for calibration uploads and wired upload UI accordingly.

• Focused processing tests

  • Added targeted tests for:
    • sub-15s aggregation behavior,
    • no-op behavior at/above 15s,
    • comma CSV parsing,
    • semicolon CSV parsing,
    • Excel calibration parsing.

def read_od_adjustment_table(file) -> pd.DataFrame:
    suffix = Path(getattr(file, "name", "")).suffix.lower()
    if hasattr(file, "seek"):
        file.seek(0)
    if suffix in {".xlsx", ".xls"}:
        return pd.read_excel(file).convert_dtypes()

    preview = file.read(4096) if hasattr(file, "read") else ""
    if hasattr(file, "seek"):
        file.seek(0)
    if isinstance(preview, bytes):
        preview = preview.decode("utf-8", errors="ignore")

    header = next((line for line in preview.splitlines() if line.strip()), "")
    try:
        delimiter = csv.Sniffer().sniff(header, delimiters=",;").delimiter
    except csv.Error:
        delimiter = ","

    return pd.read_csv(file, sep=delimiter).convert_dtypes()

Also updated:

• IQR and sliding window ranges
• make rounding less brittle (every 5 secs)

[enryH]

very good. step-for-step changes on spot.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 62e78ce9df

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Recompute rounded timestamps after raw aggregation

When high-frequency PioReactor data does not start exactly on a 15-second boundary, timestamp_localtime is floored to the aggregation bin but the already-created timestamp_rounded column is kept as the first raw sample's rounded value. process_od_pioreactor later derives start_time, elapsed seconds, and the wide pivot from timestamp_rounded, so samples like 00:00:03/:08/:13 get aggregated into the 00:00:00 bin but can still be labeled 00:00:05, shifting the time axis and any metadata alignment. Please recompute or drop timestamp_rounded after aggregation, or aggregate before creating it.

Useful? React with 👍 / 👎.