Open
Description
At the moment, if implicit multi-threading is turned on, RPageSinkBuf::CommitPage
will unconditionally copy the (uncompressed) page and create a task for sealing:
root/tree/ntuple/v7/src/RPageSinkBuf.cxx
Lines 194 to 203 in a2cb0a6
This leads to increased memory usage for high compression ratios when no spare threads are available or too slow to keep up consuming the tasks. Heuristically, we could keep track of the total number of "queued" bytes and seal a page immediately if it is higher than a threshold (approx. zipped cluster size?). This should always leave enough work for other threads to pick up, while reducing memory usage because only the compressed page is kept.