Expected results for benchmarks #10
Description
Our regression tests mostly have expected results that can be tested against, but none of our benchmarks do. This means it's impossible to know when Goblint has become better or worse on them and to track regressions in benchmarks automatically.
Various degrees of expected results would be possible:
- Explicit annotations in benchmark sources, using a common mechanism with regression tests.
- Expected statistics: race counts, warning counts (by category), etc.
- Expected resource usage: rough runtime, memory use.
- Expected comparison outcomes between configurations
- Expected comparison with previous runs