Improvements for big CI setups

[PragTob]

With #960 we got a big blocker for big CI setups (hundreds of parallel CI jobs) out of the way to use SimpleCov.collate and generate a report.

We still have plenty of optimization potential left to make SimpleCov faster and more efficient for those setups:

• allow saving of .resultset.json without going through SimpleCov::Result or reducing the overhead of SimpleCov::Result massively, Improve performance of collate & reduce memorty consumption #960 already massively reduced how often we instantiate it but as its core we only want to dump the collected coverage data into .resultset.json which we should be able to do easily without it
• make it easy to run with no formatter, right now we recommend to run with a simple formatter but it'd be even better to run with no formatter at all - all the separate CI jobs don't need to go through the overhead of generating a full report, only the final one with collate should. I'm not even sure if we can specify formatter nil or formatters [] right now.
• Package up these settings in a new profile for easy usage

[mhenrixon]

Hi @PragTob, I am interested in learning more about what I can do in the meanwhile. I upgraded to 0.21.2 because I saw performance was improved. Alas, the performance is considerably less for now. I am running the test suite locally (not in parallel) and by the SimpleCov starts to process the coverage I have an additional couple of minutes for SimpleCov to sort and format the result I guess.

I turned off branch coverage, which helped a little but the performance is still far from acceptable to me. Do you need any assistance in finding and fixing these issues? If so, can you point me to where I can start looking? I saw #916 for instance, might be a good place to start?

[PragTob]

@mhenrixon can you please open a new issue about this? With your setup and at best a repo to reproduce/where the time is spent. The changes should not have impacted normal setups that don't use SimpleCov.collate as the changes were aimed at making those faster. Of course, I might have broken stuff while changing this which of course I'd wanna change. Performance should not have digressed from 0.20 to 0.21. Or do you mean even different scenarios? Anyhow, a separate issue would be appreciated - thank you!