[Bug]: del() does not work as expected

[mrfh92]

What happened?

If you apply del() to a DNDarray, the memory is not always freed as expected

Code snippet triggering the error

Run this with 2 MPI-processes on 2 GPUs: 

import heat as ht 
import time 

x = ht.random.randn(100000,50000, split=0, dtype=ht.float32, device="gpu")

time.sleep(2)

del(x) 

time.sleep(2) 

print("Memory freed!")

Error message or erroneous outcome

One would expect that the ~10GB of data on each GPU get freed by del(x), but this is not the case as one can find out with perun (sampling_period = 0.1)

[This is the memory for the first GPU, but the second shows the same behaviour.]

[mrfh92]

With device="cpu" this looks different (with 10x the data to see the impact clearer):

At the end, the memory seems to be free.

[mrfh92]

With

del(x)
torch.cuda.empty_cache()

we get the following:

[mrfh92]

What should be much more a problem to us is that in

x = torch.zeros(50000,5000, dtype=torch.float32, device=f"cuda:{ht.MPI_WORLD.rank}")
time.sleep(2)
del(x)

also memory is not freed:

Again, one has to free manually in order to see the effect:

[Markus-Goetz]

This is not necessarily a bug, but purely a behavior of the caching memory allocator of PyTorch on GPU, https://pytorch.org/docs/stable/notes/cuda.html#memory-management

[mrfh92]

Yes, its not a bug in PyTorch, but a bug in Heat because internally we use del(...) at some points to delete objects that are no more needed and this does not have the required effect without adding torch.cuda.empty_cache().

[github-actions]

Branch bugs/1749-_Bug_del_does_not_work_as_expected created!

[mrfh92]

ok, I understand now its only a measurement problem as nvidia-smi sees that as allocated memory but PyTorch (and thus Heat) knows that its free for allocating further tensors