Sharing memory with a NumPy array

[hmaarrfk]

What did you do?

Trying to do computation on a large numpy array with pillow.

What did you expect to happen?

No memory copy to happen

What actually happened?

Memory was copied on write.

What versions of Pillow and Python are you using?

Python 3.6

# Memory benchmarking has been testing on Mac/Linux
# not sure about windows
import resource
import numpy as np
import PIL
import os
import psutil
process = psutil.Process(os.getpid())

# Load the PIL and image module so that we don't count it against the code
_ = np.zeros(1)
_ = PIL.Image.frombuffer('L', (1, 1), np.uint8(0), 'raw', 'L', 0, 1)
start_memory_usage = process.memory_info().rss
current_memory_usage = start_memory_usage
print(f'Starting memory usage {start_memory_usage/1E6:.0f} MB')

#size = (1024 * 16, 1024 * 16)
size = (1024 * 16 *2, 1024 * 16 //2)

print(f'Image should be: {size[1] * size[0] / 1E6:.0f} MB')
# approximately 250 Megabytes as

n = np.ones(size[::-1], dtype = 'uint8') * 4
previous_memory_usage = current_memory_usage
current_memory_usage = process.memory_info().rss
print(f'After creating a {n.shape[1::-1]} np array memory increased by {(current_memory_usage - previous_memory_usage)/1E6:.0f} MB')

p = PIL.Image.frombuffer('L', size, n, 'raw', 'L', 0, 1)
previous_memory_usage = current_memory_usage
current_memory_usage = process.memory_info().rss
print(f'After creating a {p.size} PIL image from array, memory increased by {(current_memory_usage - previous_memory_usage)/1E6:.0f} MB')

print(f'p[0, 0] is {p.getpixel((0, 0))}')
print(f'n[0, 0] is {n[0, 0]}')
print('setting n[0, 0] to 1')
n[0,0] = 1
print(f'p[0, 0] is {p.getpixel((0, 0))}')
print(f'n[0, 0] is {n[0, 0]}')
previous_memory_usage = current_memory_usage
current_memory_usage = process.memory_info().rss
print(f'After setting n, memory increased by {(current_memory_usage - previous_memory_usage)/1E6:.0f} MB')
print('setting p[0, 0] to 2')
p.putpixel((0, 0), 2)
print(f'p[0, 0] is {p.getpixel((0, 0))}')
print(f'n[0, 0] is {n[0, 0]}')
previous_memory_usage = current_memory_usage
current_memory_usage = process.memory_info().rss
print(f'After setting p, memory increased by {(current_memory_usage - previous_memory_usage)/1E6:.0f} MB')
print('setting n[0, 0] to 3')
n[0,0] = 3
print(f'p[0, 0] is {p.getpixel((0, 0))}')
print(f'n[0, 0] is {n[0, 0]}')

previous_memory_usage = current_memory_usage
current_memory_usage = process.memory_info().rss
print(f'After setting n again, memory increased by {(current_memory_usage - previous_memory_usage)/1E6:.0f} MB')

output

PIL version : 5.2.0
numpy version : 1.15.0
Starting memory usage 154 MB
Image should be: 268 MB
After creating a (32768, 8192) np array memory increased by 269 MB
After creating a (32768, 8192) PIL image from array, memory increased by 0 MB
p[0, 0] is 4
n[0, 0] is 4
setting n[0, 0] to 1
p[0, 0] is 1
n[0, 0] is 1
After setting n, memory increased by 0 MB
setting p[0, 0] to 2
p[0, 0] is 2
n[0, 0] is 1
After setting p, memory increased by 269 MB
setting n[0, 0] to 3
p[0, 0] is 2
n[0, 0] is 3
After setting n again, memory increased by 0 MB

I know this kind of thing is hard because we want to make sure to avoid a segfault, but it would be tremendously useful to have this feature especially when going back and forth between numpy and Pillow.

Do you have any plans to implement this?

Thank you for considering.

[yxdragon]

How does this issue go? I need this feature too, Numpy array has many processing function in (opencv, scikit-image, itk), But I did not find a plot library can draw as well as pil.draw. (scikit-image is slow, and cv did not draw no-english text, and pil 's style is very meticulous, such as poly line's junction style)

[hmaarrfk]

Through some benchmarks. I found that imageio very likely found out how to do this. Maybe try to read through imageio

[radarhere]

See also #2980

[homm]

Have you tried to set p.readonly = False?

[hmaarrfk]

Hmm you are right, adding p.readonly = False seems to do the trick

# Memory benchmarking has been testing on Mac/Linux
# not sure about windows
import resource
import numpy as np
import PIL
import os
import psutil
process = psutil.Process(os.getpid())

# Load the PIL and image module so that we don't count it against the code
_ = np.zeros(1)
_ = PIL.Image.frombuffer('L', (1, 1), np.uint8(0), 'raw', 'L', 0, 1)
start_memory_usage = process.memory_info().rss
current_memory_usage = start_memory_usage
print(f'Starting memory usage {start_memory_usage/1E6:.0f} MB')

#size = (1024 * 16, 1024 * 16)
size = (1024 * 16 *2, 1024 * 16 //2)

print(f'Image should be: {size[1] * size[0] / 1E6:.0f} MB')
# approximately 250 Megabytes as

n = np.ones(size[::-1], dtype = 'uint8') * 4
previous_memory_usage = current_memory_usage
current_memory_usage = process.memory_info().rss
print(f'After creating a {n.shape[1::-1]} np array memory increased by {(current_memory_usage - previous_memory_usage)/1E6:.0f} MB')

p = PIL.Image.frombuffer('L', size, n, 'raw', 'L', 0, 1)
p.readonly = False   # <<<<< Added this line
previous_memory_usage = current_memory_usage
current_memory_usage = process.memory_info().rss
print(f'After creating a {p.size} PIL image from array, memory increased by {(current_memory_usage - previous_memory_usage)/1E6:.0f} MB')

print(f'p[0, 0] is {p.getpixel((0, 0))}')
print(f'n[0, 0] is {n[0, 0]}')
print('setting n[0, 0] to 1')
n[0,0] = 1
print(f'p[0, 0] is {p.getpixel((0, 0))}')
print(f'n[0, 0] is {n[0, 0]}')
previous_memory_usage = current_memory_usage
current_memory_usage = process.memory_info().rss
print(f'After setting n, memory increased by {(current_memory_usage - previous_memory_usage)/1E6:.0f} MB')
print('setting p[0, 0] to 2')
p.putpixel((0, 0), 2)
print(f'p[0, 0] is {p.getpixel((0, 0))}')
print(f'n[0, 0] is {n[0, 0]}')
previous_memory_usage = current_memory_usage
current_memory_usage = process.memory_info().rss
print(f'After setting p, memory increased by {(current_memory_usage - previous_memory_usage)/1E6:.0f} MB')
print('setting n[0, 0] to 3')
n[0,0] = 3
print(f'p[0, 0] is {p.getpixel((0, 0))}')
print(f'n[0, 0] is {n[0, 0]}')

previous_memory_usage = current_memory_usage
current_memory_usage = process.memory_info().rss
print(f'After setting n again, memory increased by {(current_memory_usage - previous_memory_usage)/1E6:.0f} MB')

Starting memory usage 137 MB
Image should be: 268 MB
After creating a (32768, 8192) np array memory increased by 269 MB
After creating a (32768, 8192) PIL image from array, memory increased by 0 MB
p[0, 0] is 4
n[0, 0] is 4
setting n[0, 0] to 1
p[0, 0] is 1
n[0, 0] is 1
After setting n, memory increased by 0 MB
setting p[0, 0] to 2
p[0, 0] is 2
n[0, 0] is 2
After setting p, memory increased by 0 MB
setting n[0, 0] to 3
p[0, 0] is 3
n[0, 0] is 3
After setting n again, memory increased by 0 MB

[radarhere]

Does p.readonly = False mean that this issue is resolved?