Skip to content

Commit 32381af

Browse files
committed
BENCH: profile memleaks asv
* add some `asv` peak memory benchmarks as regression guards against gh-811; the only thing missing (IMO) here to completely close gh-817 is a version of the log file with `HEATMAP` in it (see the diff comments for why I don't have this for `asv` suite at the moment) * although this won't automatically run in CI because it takes too long and too much memory, it still seems to do an excellent job of being sensitive to the patched leaks over the recent history of our project: - `asv continuous -e -b "peakmem.*" b6c8599 main` ``` before after ratio [b6c8599] [3fb2c36] <treddy_memleak_cffi_only_pt2~1> <main> - 239M 76M 0.32 cffi_backend.RetrieveLogData.peakmem_log_get_record('LUSTRE') - 241M 76M 0.32 cffi_backend.RetrieveLogData.peakmem_log_get_record('H5F') - 257M 75.8M 0.30 cffi_backend.RetrieveLogData.peakmem_log_get_record('STDIO') - 289M 76M 0.26 cffi_backend.RetrieveLogData.peakmem_log_get_record('MPI-IO') - 303M 76M 0.25 cffi_backend.RetrieveLogData.peakmem_log_get_record('POSIX') - 304M 75.4M 0.25 cffi_backend.RetrieveLogData.peakmem_log_get_record('DXT_POSIX') - 304M 75.3M 0.25 cffi_backend.RetrieveLogData.peakmem_log_get_record('DXT_MPIIO') - 324M 75.8M 0.23 cffi_backend.RetrieveLogData.peakmem_log_get_record('H5D') - 23.7G 75.4M 0.00 cffi_backend.RetrieveLogData.peakmem_log_get_mounts('DXT_POSIX') - 23.7G 75.4M 0.00 cffi_backend.RetrieveLogData.peakmem_log_get_mounts('H5F') - 23.7G 75.4M 0.00 cffi_backend.RetrieveLogData.peakmem_log_get_mounts('H5D') - 23.7G 75.4M 0.00 cffi_backend.RetrieveLogData.peakmem_log_get_mounts('STDIO') - 23.7G 75.4M 0.00 cffi_backend.RetrieveLogData.peakmem_log_get_mounts('MPI-IO') - 23.7G 75.4M 0.00 cffi_backend.RetrieveLogData.peakmem_log_get_mounts('DXT_MPIIO') - 23.7G 75.4M 0.00 cffi_backend.RetrieveLogData.peakmem_log_get_mounts('LUSTRE') - 23.7G 75.4M 0.00 cffi_backend.RetrieveLogData.peakmem_log_get_mounts('POSIX') SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY. PERFORMANCE INCREASED. ```
1 parent fb6bb27 commit 32381af

File tree

1 file changed

+33
-0
lines changed

1 file changed

+33
-0
lines changed
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
from darshan.backend import cffi_backend
2+
from darshan.log_utils import get_log_path
3+
4+
5+
class RetrieveLogData:
6+
# see gh-811
7+
8+
# TODO: maybe produce a version of the log that also
9+
# has runtime HEATMAP to guard memory leaks on that module;
10+
# part of the issue here is
11+
# perhaps the following to pull in new log repo files:
12+
# https://github.com/darshan-hpc/darshan-logs/issues/15
13+
14+
params = ['POSIX', 'MPI-IO', 'H5F', 'H5D', 'LUSTRE', 'STDIO', 'DXT_POSIX', 'DXT_MPIIO']
15+
param_names = ["module"]
16+
17+
18+
def setup(self, module):
19+
self.ior_log_path = get_log_path("ior_hdf5_example.darshan")
20+
21+
22+
def peakmem_log_get_record(self, module):
23+
for i in range(100000):
24+
log = cffi_backend.log_open(self.ior_log_path)
25+
rec = cffi_backend.log_get_record(log, module)
26+
cffi_backend.log_close(log)
27+
28+
29+
def peakmem_log_get_mounts(self, module):
30+
for i in range(100000):
31+
log = cffi_backend.log_open(self.ior_log_path)
32+
rec = cffi_backend.log_get_mounts(log)
33+
cffi_backend.log_close(log)

0 commit comments

Comments
 (0)