Output file compression settings

[pieterdavid]

My plotters ran slowly yesterday, so I started a few tests to see if there are easy gains possible... I see here that there is some optimization for the basket size etc.
Would these settings interfere with the compression settings? We seem to be currently using ZLIB level 1, with LZMA level 7 the file size goes down by about 35% (compression factor from 3.5 to more than 5). This would be a fairly aggressive setting (uncompressing costs quite some CPU), but I think the plotters are usually limited by IO anyway - I am running some more tests to verify that.

[OlivierBondu]

IIRC we played with these to limit memory consumption at run time, maybe it affects the compression but I think it shouldn't...

[pieterdavid]

Thanks, I'll take that one into account as well

[pieterdavid]

Decompressing 5GB with LZMA adds about 2min of CPU time... that doesn't seem excessive to me, but given how little the plotters need (a bit over 1min in this case) this would make them slower when load on the storage is low. I will try to collect a bit more statistics from slurm and submit a PR to make it configurable, but not change the default (yet), shout if you disagree ;-) .