Skip to content

feat: add include_failed_runs benchmark override - #147

Merged
edmundmiller merged 2 commits into
mainfrom
hermes/include-failed-runs
Apr 23, 2026
Merged

feat: add include_failed_runs benchmark override#147
edmundmiller merged 2 commits into
mainfrom
hermes/include-failed-runs

Conversation

@edmundmiller

Copy link
Copy Markdown
Contributor

Summary

  • add an include_failed_runs / include-failed-runs override for benchmark report generation
  • keep the current default behavior unchanged: failed workflows remain excluded unless the flag is set
  • continue excluding cancelled workflows
  • plumb the flag through the Python CLIs, Nextflow params, schema, and workflow wiring
  • add regression coverage for the new override behavior

Why

When comparing benchmark runs, it is useful to inspect workflow-level failed runs in the downstream HTML sections and charts instead of only seeing them dimmed/excluded in the run summary.

Test plan

  • UV_NO_CONFIG=1 uv run --with typer --with pyyaml --with jinja2 --with pyarrow --with pytest --with httpx pytest -q modules/local/aggregate_benchmark_report_data/tests/test_aggregate.py::test_failed_and_cancelled_runs_only_appear_in_overview modules/local/aggregate_benchmark_report_data/tests/test_aggregate.py::test_include_failed_runs_override_includes_failed_in_downstream_sections
  • UV_NO_CONFIG=1 uv run --with typer --with pyyaml --with jinja2 --with pyarrow --with pytest --with httpx pytest -q modules/local/aggregate_benchmark_report_data/tests/test_aggregate.py modules/local/render_benchmark_report/tests/test_render.py
  • regenerated a local HTML report with failed runs included and verified the failed run IDs appear in downstream sections

Notes

  • this change does not address the separate latest-main regressions around raw tw runs dump tarballs or the aggregate-stage symlink/import issue
  • this PR focuses only on the include-failed-runs override

@FloWuenne

Copy link
Copy Markdown
Contributor

@copilot resolve the merge conflicts in this pull request

…-runs

# Conflicts:
#	README.md
#	nextflow.config
#	nextflow_schema.json

Co-authored-by: FloWuenne <11639154+FloWuenne@users.noreply.github.com>

Copilot AI commented Apr 23, 2026

Copy link
Copy Markdown
Contributor

@copilot resolve the merge conflicts in this pull request

Resolved and pushed. I merged origin/main into this branch, resolved the conflicts in README.md, nextflow.config, and nextflow_schema.json, and preserved both feature sets. Commit: 565e9e3.

Copilot AI requested a review from FloWuenne April 23, 2026 13:52
@FloWuenne
FloWuenne marked this pull request as ready for review April 23, 2026 13:55
@edmundmiller
edmundmiller merged commit 703f526 into main Apr 23, 2026
55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants