You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: .seqera/context/ERRORS.md
-1Lines changed: 0 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -61,4 +61,3 @@ Both runs `1cF65l2PDvxNd5` (maniac_celsius) and `5VGC0gjEmghOaz` (clever_kalman)
61
61
-**No failed runs analysed:** Both Platform runs are currently successful. To populate this section with real failure analysis, re-run context generation after a failure occurs or provide historical failed run IDs.
62
62
-**No STAR/alignment task data yet:** Both runs are still in early stages (QC/trim). Resource pressure and potential OOM failures during STAR genome generation and alignment are not yet observable.
63
63
-**No cost data:** Neither run has completed, so total run cost is unavailable. Individual task costs are in the $0.001–$0.007 range for early QC tasks.
The timing variance in short tasks (SAMTOOLS_FAIDX: 28s vs 62s) likely reflects spot instance variability and S3 staging latency rather than algorithmic differences. Both runs are still in progress — STAR genome generation and alignment phases are pending.
63
61
64
62
## Local Development History (git log)
@@ -69,4 +67,3 @@ Recent development has focused on the **benchmark report rendering pipeline**:
69
67
2.**Combined runtime charts** (18a64f4 → a49a9a4): Added combined task runtime visualisation with scheduling overhead, per-pipeline breakdowns, and ECharts integration
70
68
3.**CUR cost streaming** (b96ecfd): Optimised AWS CUR parquet processing with batch streaming
71
69
4.**Test colocation** (96b8b81): Moved benchmark tests to live alongside their modules
CSV with columns: `id` (required), `workspace` (required, `org/name` or `external`), `group` (optional), `logs` (path to tarball/dir for external), `platform` (per-row API URL override), `token_env` (per-row env var name for bearer token).
@@ -99,4 +93,3 @@ results/
99
93
3.**All JSON files** are collected into a single temp directory and passed to the 3-stage Python pipeline: normalize → aggregate → render.
100
94
4. The Python stages are separate Nextflow processes sharing one Wave container image (`python_duckdb_jinja2_typer_pruned`).
101
95
5. JSONL is the handoff format — streaming-friendly for large run datasets and Fusion FS compatible.
Copy file name to clipboardExpand all lines: modules/local/aggregate_benchmark_report_data/AGENTS.md
+8Lines changed: 8 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,26 +1,32 @@
1
1
# aggregate_benchmark_report_data
2
2
3
3
Purpose
4
+
4
5
- Convert the JSONL bundle into a single `report_data.json` document for rendering.
5
6
6
7
Owns
8
+
7
9
-`AGGREGATE_BENCHMARK_REPORT_DATA` in `main.nf`
8
10
- Module-local CLI entrypoint in `bin/benchmark_report.py`
9
11
- Shared aggregation logic in repo-root `bin/benchmark_report_aggregate.py`
10
12
- Stage-scoped tests under `tests/`
11
13
12
14
Run directly
15
+
13
16
-`nextflow run modules/local/aggregate_benchmark_report_data/main.nf -profile docker,arm --jsonl_bundle <jsonl_bundle_dir>`
14
17
- The direct `nextflow run modules/local/...` path depends on the module-local `bin/benchmark_report.py` shim/CLI; keep that invocation shape working if you refactor the stage.
15
18
16
19
Inputs
20
+
17
21
-`jsonl_bundle/` from `normalize_benchmark_jsonl`
18
22
19
23
Outputs
24
+
20
25
-`report_data.json`
21
26
-`versions.yml`
22
27
23
28
Invariants
29
+
24
30
- This is the boundary between streaming records and presentation data.
0 commit comments