Skip to content

Conversation

@saurabh12nxf
Copy link

Summary

This PR implements periodic writes for task-set-specific result files when using --results-per-taskset, addressing issue #1215.

Previously, task-set files were only written at the end of benchmark execution, which meant:

  • No intermediate results were available during long-running benchmarks
  • All results were lost if BenchExec crashed before completion

This change makes task-set files behave consistently with rundefinition files by writing them periodically (every 60 seconds) during execution.

Changes Made

  • benchexec/outputhandler.py:
    • Added tracking for task-set (block) XML files in output_before_run_set() method
    • Implemented periodic write logic in output_after_run() method
    • Files are written incrementally using _write_rough_result_xml_to_file() every 60 seconds
    • Final files are written with proper formatting using _write_pretty_result_xml_to_file() at the end

Benefits

  1. Crash resilience: Results are preserved even if BenchExec crashes
  2. Progress monitoring: Users can view partial results during execution
  3. Consistent behavior: Task-set files now behave like rundefinition files
  4. Minimal overhead: Writes occur at most once per 60 seconds per task-set
  5. Backward compatible: No changes to existing behavior when both file types are enabled

Testing

  • ✅ Syntax and import checks passed
  • ✅ Existing unit tests pass (no regressions)
  • ✅ Functional testing with example benchmarks confirms task-set files are created and updated periodically
  • ✅ XML structure verified to be correct

Implementation Details

The implementation follows existing patterns in the codebase:

  • Uses the same 60-second interval as rundefinition files
  • Maintains thread safety with existing print_lock mechanism
  • Handles edge cases (duplicate block names, empty blocks)
  • Reuses existing methods (runs_to_xml, _write_rough_result_xml_to_file)

Closes #1215

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Periodic writes to task-set-specific result files

1 participant