Description
Right now the documentation for troubleshooting build failures has this as a first, required step:
Configure BenchmarkDotNet to keep auto-generated benchmark files (they are being removed after benchmark is executed by default).
And all the methods of doing this appear to require me to locate the benchmark in question and then modify its source code intrusively, which will get blown away if I revert my git checkout etc. Anyone else troubleshooting will also need to do this. Maybe I don't understand BDN well enough and there's some obvious way to feed --keepfiles through without modifying the code that calls BenchmarkSwitcher, like using a magic environment variable?
Anyway, I think the default should be to keep the directory around, or at a minimum there should be an environment var you can set to produce this. In the first place, it says 'removed after benchmark is executed', but that isn't an accurate characterization of this behavior, since the benchmark never executed (due to the build failing). It took me a while to even figure out that my attempts at getting a binlog out of this were working, and that BDN was just deleting the logs. I can imagine some disk space usage issues if someone runs thousands of failed benchmark builds (and accumulates temporary output directories), but that seems like a corner case with multiple easy solutions.