frombuffer constructed images are disconnected from the original data

[dov]

What did you do?

I wanted to use PIL to render into an externally allocated buffer without any memcpy, and thought that the frombuffer constructor would allow me to do that.

To test this I first created a np.array and then an image with the Image.frombuffer() and expected the reference to the data to be bidirectional (the documentation isn't clear on that). I.e. I expected that I can change a numpy array by creating an image based on it's data and then using paint operators.

This did not work though. Is this a bug or a feature? Is there any particular reason why this isn't supported?

See below for a code example.

What did you expect to happen?

I expected the assert in the code below to succeed. I.e. that the draw.rectangle() call would change the data in the data variable.

What actually happened?

The assert failed.

What versions of Pillow and Python are you using?

4.3.0

Please include code that reproduces the issue and whenever possible, an image that demonstrates the issue. Please upload images to GitHub, not to third-party file hosting sites. If necessary, add the image to a zip or tar archive.

The best reproductions are self-contained scripts with minimal dependencies. If you are using a framework such as plone, Django, or buildout, try to replicate the issue just using Pillow.

from PIL import Image
from PIL import ImageDraw
import numpy as np

width,height=1,8
data = np.array([ord('A')]*(width*height),dtype=np.uint8)            #  An 1x8 chunk composed of 'A'

img = Image.frombuffer('L',(width,height),
                       data,'raw','L',0,1)
assert(img.getpixel((0,0)) == ord('A'))     # Assert that the data to
                                            # img conversion works
draw = ImageDraw.Draw(img)
draw.rectangle((0,0,width,height),ord('B')) # Draw B's in the entire image
assert(img.getpixel((0,0)) == ord('B'))     # Assert that the drawing worked
print 'data=',data
assert(data[0] == ord('B'))                 # Assert that the underlying
                                            # data was changed

[wiredfool]

The memory map that Pillow performs is a read-only map. At the point where the draw command happens, Pillow copies the buffer into a read/write image object.

from PIL import Image
from PIL import ImageDraw
import numpy as np

width,height=1,8
data = np.array([ord('A')]*(width*height),dtype=np.uint8)            #  An 1x8 chunk composed of 'A'

img = Image.frombuffer('L',(width,height),
                       data,'raw','L',0,1)
assert(img.getpixel((0,0)) == ord('A'))     # Assert that the data to
                                            # img conversion works

print (img.readonly)
data[0] = 67
assert(img.getpixel((0,0)) == 67)
print (img.readonly)

draw = ImageDraw.Draw(img)
draw.rectangle((0,0,width,height),ord('B')) # Draw B's in the entire image
print (img.readonly)
assert(img.getpixel((0,0)) == ord('B'))     # Assert that the drawing worked

yeilds:

(vpy27) erics@builder-16:~/test$ python issue_2973.py 
1
1
0

Prior to that draw command, the img buffer is still connected to the underlying np.array, as the write to the data does extend into the image.

[dov]

Ok. I understand. Is there any particular reason why the memory is copied when drawing? Why not write into the original memory? I thought that was the whole point of the buffer protocol was to share data?

Compare e.g. with the corresponding pure numpy script which remains connected even after writing into the copy:

import numpy as np

width,height=1,8
data = np.array([65]*8, dtype=np.uint8)
a = np.frombuffer(data.data, dtype=np.uint8)
print a
a[0] = ord('B')
print a
print data
assert(a[0]==data[0])

Output:

[65 65 65 65 65 65 65 65]
[66 65 65 65 65 65 65 65]
[66 65 65 65 65 65 65 65]

[dov]

Thanks for fixing this! I'll close this bug thanks to #2980.