I was trying to compare benchmark results for A100 PCI and A100 SXM, but nvbench refused with:
nvbench_compare.py ./babelstream_fallback_blocks_A100_PCI/ ./babelstream_fallback_blocks_A100_SXM/
['./babelstream_fallback_blocks_A100_PCI/', './babelstream_fallback_blocks_A100_SXM/']
Device sections do not match.
I can understand that you are trying to prevent users from mixing up results from different GPUs, but I would really like the ability to compare a benchmark across devices. Could the error be demoted to a warning maybe?