Release 3.34

• One new tool-info module.

Release 3.33

• One new tool-info module.

Release 3.32

• Updated tool-info modules for SV-COMP'26 and Test-Comp'26.

Release 3.31

BenchExec now requires Python 3.10 or newer.

• Provide command-line arguments for choosing result files of benchexec.
  So far benchexec writes result files for every <rundefinition> and <tasks> tag. This can be convenient but also redundant, and there is an inconsistency in case of a run definition with a single task set. To solve this, we introduce two new command-line arguments:

  • --results-per-rundefinition
  • --results-per-taskset

  Using them allows one to choose the desired result files, either only one set of files or both. If none of the arguments is given the previous default behavior is used, though in BenchExec 4.0 we will change the default behavior to a more consistent one.

• Duplicate names of <rundefinition> tags in the benchmark definition are now forbidden.
  It was an oversight that this was allowed, because it would lead to output files with duplicate names, i.e., some results overwriting other results. Now benchexec detects and prevents this. Until the next major release duplicate run-definition names are still allowed in cases where it does not cause actual problems (e.g., because only one of them is selected for execution). In BenchExec 4.0 we will likely forbid such duplicate names completely.

• Prevent overwriting results when <tasks> tags with duplicate names exist in the benchmark definition.
  Such cases lead to output files with duplicate names, i.e. some results overwriting other results. Now benchexec detects this, warns about it, and skips writing the files with duplicate names. In BenchExec 4.0 we will likely forbid such duplicate names completely.

• Fix crash on VMs where the CPU frequency is unknown.
  It seems this happens on some VMs on ARM Macs. Thanks to @leventeBajczi for reporting and fixing this!

• Fix crash in special cases of cgroupsv2 systems with a memory limit defined outside of BenchExec (e.g., by the system administrator).

• Fix crash in case of running BenchExec inside a container with an incomplete cgroupsv2 setup where we are in an out-of-tree cgroup.

• Some performance improvements for large benchmark definitions that include task definitions more than once.

• Many new and improved tool-info modules.

Release 3.30

This will be the last release of BenchExec to support Python 3.8 and 3.9.
Future versions will require Python 3.10.

• Rename statistics row "local summary" in HTML tables to "summary measurements" and improve tooltip explanation to make it better understandable.
  Also add a command-line argument --no-local-summary to table-generator to be able to hide it completely.
• Fix crash on cgroupv2 systems where some cgroupsv1 controllers are not present in the kernel anymore (e.g., Fedora 41 or newer).
  Thanks to our GSoC participant @t0hsumi for investigating and fixing this!
• Fix visibility of bind mounts in container mode with fuse-overlayfs.
• Improved explanation in error message about forbidden user namespaces.
• Some new and improved tool-info modules.

Note: We now recommend all users of BenchExec on modern systems (kernel 5.19 or newer) to use cgroupsv2 instead of the outdated cgroupsv1.

Release 3.29

• The name of the wheel and tar archives in BenchExec releases changes from BenchExec-$VERSION* to benchexec-$VERSION* in accordance with Python standard PEP 625 and PyPI requirements.
• Some new and improved tool-info modules.

Release 3.28

• Some new and improved tool-info modules.

Release 3.27

• Support for version 2.1 of the task-definition format.
• Updated tool-info modules for SV-COMP'25 and Test-Comp'25.

Release 3.26

This release brings several important fixes for cgroups v2 and all users on systems with cgroups v2 are strongly recommended to upgrade.

• Fix regression from BenchExec 3.25 for detecting memory-limit violations of a run.
  The changes related to the LXCFS integration in the last version caused the problem that a run that hit the memory limit was no longer marked as "out of memory" but as "failed" and did not have any measurement results (only for cgroups v2 and with container mode enabled).
  Note that the memory limit was still correctly working and also measurements produced for non-failed runs were correct.
• Fix AssertionError when creating a systemd scope for cgroups v2 usage.
  This was caused by a race condition and did occur only with a certain probability depending on the system load.
• Fix warning about left-over cgroup during run cleanup for cgroups v2.
• More robust cleanup of cgroups created by the benchmarked process.
• Fix installation of our Python package on Windows.
• Several new and improved tool-info modules.

Release 3.25

• Support for fuse-overlayfs as alternative to kernel-based overlayfs
  BenchExec uses overlayfs for providing a virtualized file-system in its containers (unless configured otherwise by the user). Unfortunately, the overlayfs implementation in the Linux kernel does not support all use cases such as overlayfs for the root directory or triple-nested containers (cf. #776 and #1067). Now BenchExec makes use of the alternative implementation fuse-overlayfs if installed in version 1.10 or newer as an automatic fallback when necessary. This makes BenchExec work again in its default configuration without requiring parameters like --read-only-dir /.
  We are glad about this long-awaited feature being contributed by GSoC participant @younghojan! Thanks!
• Improve LXCFS integration
  If installed, BenchExec uses LXCFS to provide a better virtualization of the environment visible inside the container, for example by virtualizing the uptime. Now we use LXCFS also to virtualize CPU information in /proc/cpuinfo and /sys/devices/system/cpu. This allows the benchmarked process to see more easily how many CPU cores they are allowed to use.
• New tool-info module for super_prove.