Issue accessing modules and variables defined outside function on Windows with Pool.map

[astrofrog]

When using multiprocess.Pool, I am getting a different behavior on Mac/Linux compared to Windows - on Windows the following example does not work:

In [1]: import os                                                                                                                                                                                                                               
In [2]: def example(filename):
...:     return os.path.abspath(filename)                                                                              
In [3]: from multiprocess import Pool                                                                                                                                                                                                           
In [4]: p = Pool()
 ...: p.map(example, ['a.jpg', 'b.jpg', 'c.jpg'])                                                                     
---------------------------------------------------------------------------                                             
RemoteTraceback                          
 Traceback (most recent call last)                                             
RemoteTraceback:                                                                                                        
"""                                                                                                                     
Traceback (most recent call last):                                                                                        
File "C:\Users\Thomas Robitaille\.conda\envs\py311\Lib\site-packages\multiprocess\pool.py", line 125, in worker          
 result = (True, func(*args, **kwds))                                                                                                    
^^^^^^^^^^^^^^^^^^^                                                                                   
File "C:\Users\Thomas Robitaille\.conda\envs\py311\Lib\site-packages\multiprocess\pool.py", line 48, in mapstar           
return list(map(*args))                                                                                                        
^^^^^^^^^^^^^^^^                                                                                              
 File "<ipython-input-2-36e5e27f9cec>", line 2, in example                                                             
NameError: name 'os' is not defined                                                                                    
 """

But this does appear to work properly on Mac and Linux by default. Is this expected behavior or a bug? Do I need to redefine all imports and variables inside the function?

[mmckerns]

As you point out, a workaround that generally works is to define the imports inside the function. The issue is likely due to Windows having a different pickler and a different serialization context than mac and linux. How well this works depends several other things as well. On Windows, it can help to use freeze_support along with a call from __main__. In certain cases, it can help to change how the global dictionary is handled (i.e. dill.settings['recurse'] = True). It can help to update your version of dill (I didn't see what version you are using). I'd consider it a known behavior. It can occur on a mac or linux as well, especially if one changes the serialization context (i.e. multiprocess.context).

Let me know if this answers your question.