Open
Description
Pickling a large boost histogram results in a second copy being created in memory. Via the use of the PickleBuffer mechanism it should be possible to avoid this.
e.g.
import boost_histogram as bh
axis = bh.axis.Regular(1024*1024*1024, 0., 1)
htest = bh.Histogram(axis)
uses 8GB of memory as expected
import boost_histogram as bh
import pickle
axis = bh.axis.Regular(1024*1024*1024, 0., 1)
htest = bh.Histogram(axis)
with open("test.pkl", "wb") as f:
pickle.dump(htest, f, protocol = pickle.HIGHEST_PROTOCOL)
uses 16GB
Metadata
Metadata
Assignees
Labels
No labels