TypeError: can't pickle cairo.Matrix objects

[idlethread]

I'm using the library to render a png but I see the following error. It seems to be on the exact line that was changed in SHA: 2601ae8. Any hints on what this means?

ctx.render_layers(pcb.top_layers, render_p + '/renders/' + name + '_top.png')

File "/home/idle/.local/lib/python3.7/site-packages/pcb_tools-0.1-py3.7.egg/gerber/render/cairo_backend.py", line 130, in render_layers
File "/home/idle/.local/lib/python3.7/site-packages/pcb_tools-0.1-py3.7.egg/gerber/render/cairo_backend.py", line 98, in render_layer
File "/home/idle/.local/lib/python3.7/site-packages/pcb_tools-0.1-py3.7.egg/gerber/render/cairo_backend.py", line 196, in _render_layer
File "/home/idle/.local/lib/python3.7/site-packages/pcb_tools-0.1-py3.7.egg/gerber/render/cairo_backend.py", line 535, in new_render_layer
File "/usr/lib/python3.7/copy.py", line 96, in copy
rv = reductor(4)
TypeError: can't pickle cairo.Matrix objects

[phsilva]

Hi @idlethread, are you using any multiprocessing in the example? Can you share the script you are runnnig?

[idlethread]

No multiprocessing. Script is effectively as follows. What I found suspicious was the change in SHA: 2601ae8 was exactly the line I see the bug.

# Extracting pcb file from directory for rendering
pcb = PCB.from_directory(gerber_folder)
blue_theme = theme.Theme(
                    topmask=RenderSettings(blue, alpha=0.8, invert=True),
                    bottommask=RenderSettings(blue, alpha=0.8, invert=True),)

#blue
ctx.render_layers(pcb.top_layers, name + '_top_blue.png', theme=blue_theme)
ctx.render_layers(pcb.bottom_layers, name + '_bottom_blue.png', theme=blue_theme)

[idlethread]

@phsilva Here is an easier way to reproduce, at least on my 19.10 Ubuntu machine:

1. Download https://kitspace.org/boards/github.com/kitspace-forks/bus_pirate/Bus_Pirate-0ca376a-gerbers.zip
2. Unzip the gerbers
3. Install pcb-tools dependencies using pip3 install -r requirements.txt
4. Install pcb-tools using python3 setup.py install --user
5. Run gerber-render BusPirate-v3.6a-SSOP.gko
Traceback (most recent call last): File "/home/amit/.local/bin/gerber-render", line 11, in <module> load_entry_point('pcb-tools==0.1', 'console_scripts', 'gerber-render')() File "/home/amit/.local/lib/python3.7/site-packages/pcb_tools-0.1-py3.7.egg/gerber/__main__.py", line 116, in main File "/home/amit/.local/lib/python3.7/site-packages/pcb_tools-0.1-py3.7.egg/gerber/render/cairo_backend.py", line 98, in render_layer File "/home/amit/.local/lib/python3.7/site-packages/pcb_tools-0.1-py3.7.egg/gerber/render/cairo_backend.py", line 196, in _render_layer File "/home/amit/.local/lib/python3.7/site-packages/pcb_tools-0.1-py3.7.egg/gerber/render/cairo_backend.py", line 535, in new_render_layer File "/usr/lib/python3.7/copy.py", line 96, in copy rv = reductor(4) TypeError: can't pickle cairo.Matrix objects

[idlethread]

@hamiltonkibbe @phsilva any suggestions on how to debug this?

Reverting 2601ae8 make it work without the error but I do see a reversed layer.

[ingobecker]

It seems like copy.copy() isn't working with the Matrix class from pycairo.

import cairo
import cairocffi
import copy

copy.copy(cairocffi.Matrix()) # works
copy.copy(cairo.Matrix()) # doesn't work

To solve your problem, make sure you don't have pycairo installed. A better solution would be to make the new_render_layer() function compatible with both cairocffi as well as pycairo by not using copy.copy() on Matrix.

[idlethread]

Hi Ingo,

Thanks for your reply.

With cairoffi, I see the following:

$ python
Python 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cairocffi
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in
_load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File
"/home/amit/.local/lib/python3.8/site-packages/cairocffi-0.6-py3.8.egg/cairocffi/__init__.py",
line 105, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in
_load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File
"/home/amit/.local/lib/python3.8/site-packages/cairocffi-0.6-py3.8.egg/cairocffi/xcb.py",
line 16, in <module>
  File "/usr/lib/python3/dist-packages/cffi/api.py", line 527, in include
    raise TypeError("ffi.include() expects an argument that is also of"
TypeError: ffi.include() expects an argument that is also of type cffi.FFI,
not 'CompiledFFI'
>>>

With cairo, I see the following:

$ python
Python 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cairo
>>> import copy
>>> copy.copy(cairo.Matrix())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/copy.py", line 92, in copy
    rv = reductor(4)
TypeError: cannot pickle 'cairo.Matrix' object

I'm afraid I'm not too well versed in the nuances of these two libraries.
This is on Ubuntu 20.04. Does the README.md or requirement.txt need updates
to update which libraries must be installed?

Regards,
Amit

[Ash-Labs]

Any solutions ?

[idlethread]

@Ash-Labs Ingo attached a patch in #165. It's not been integrated yet though. Development on pcb-tools seems to be auto-pilot, I guess.

[Ash-Labs]

@idlethread i forked and patched this. works! thanks for the prompt response!

[ingobecker]

As an alternative to patching you could also try to coerce the usage of the cairo version that works using dependency managent.

[idlethread]

@ingobecker How does one do that? See the output I get in #116 (comment)

[Ash-Labs]

As an alternative to patching you could also try to coerce the usage of the cairo version that works using dependency managent.

Since this package is in auto mode, the fork and self patch path is more stable on the long run. Code changing or version_requirements updating are both options then.

[ingobecker]

I guess the problem is this line here which prefers pycairo over cairocffi if installed. That's also a bit strange because setup.py specifies cairocffi as a requirement. If you make sure you have no pycairo installed pcb-tools will use cairocffi and will work as expected. You can do so, by using something like a virtual env or a container.

[ingobecker]

In other words, it looks like you have a version of pycairo installed. That might be the case, if you are using a global python version with all python packages installed globally. In this case, pcb-tools will have access to pycairo and use it. To prevent that behaviour and isolate the dependencies of your code you normally use something like virtual envs or a container. If you use pycairo in the same project, it might be impossible to use pcb-tools side by side with the code that requires pycairo and you have to use the patch instead.