Skip to content

[Fix] Combine split eval results in default summarizer#2451

Open
yhzhu99 wants to merge 1 commit into
open-compass:mainfrom
yhzhu99:fix-summarizer-split-results
Open

[Fix] Combine split eval results in default summarizer#2451
yhzhu99 wants to merge 1 commit into
open-compass:mainfrom
yhzhu99:fix-summarizer-split-results

Conversation

@yhzhu99

@yhzhu99 yhzhu99 commented May 15, 2026

Copy link
Copy Markdown

Summary

  • combine numbered split result files in the default summarizer when the unsplit result file is absent
  • weight merged numeric metrics by sample counts from result metadata or legacy details
  • record per-result sample counts from OpenICLEvalTask for future split runs without details

Fixes #2163

Tests

  • python3 -m flake8 tests/summarizers/test_default.py opencompass/tasks/openicl_eval.py opencompass/utils/result.py
  • python3 -m py_compile opencompass/summarizers/default.py opencompass/tasks/openicl_eval.py opencompass/utils/result.py tests/summarizers/test_default.py
  • python3 -m pytest tests/summarizers/test_default.py (blocked locally: missing torch runtime dependency)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] OpenICLEvalTask does not combine results split by NumWorkerPartitioner

2 participants