-
Notifications
You must be signed in to change notification settings - Fork 447
Description
macOS/Windows incompatibility: max_workers parameter not supported by multiprocess.Pool
Problem
Code using the max_workers parameter with the multiprocessing compatibility layer fails on macOS and Windows platforms. The multiprocess.Pool (used on darwin/win32) expects a processes parameter instead of max_workers, which breaks cross-platform compatibility.
Root Cause
The codechecker_common.compatibility.multiprocessing module currently exports multiprocess.Pool directly on macOS/Windows platforms, which has a different API than ProcessPoolExecutor used on Linux:
- macOS/Windows:
multiprocess.Pool(processes=N) - Linux:
ProcessPoolExecutor(max_workers=N)
This API mismatch prevents code from using max_workers consistently across all platforms.
Background
The multiprocess package is used on macOS/Windows instead of the standard library multiprocessing due to reliability issues (see commits 803e446, d56e216):
- On macOS: Standard
multiprocessinghas known issues - On Windows:
ProcessPoolExecutorfails due to fork() limitations and PYTHONPATH issues - On Linux:
ProcessPoolExecutorworks reliably
Expected Behavior
Code should be able to use max_workers parameter consistently across all platforms:
from codechecker_common.compatibility.multiprocessing import Pool
with Pool(max_workers=4) as pool:
results = pool.map(func, items)Actual Behavior
On macOS/Windows, using max_workers fails because multiprocess.Pool doesn't accept this parameter:
Traceback (most recent call last):
File "/Users/efulop/codechecker/venv/lib/python3.14/site-packages/codechecker_common/cli.py", line 259, in main
sys.exit(args.func(args))
~~~~~~~~~^^^^^^
File "/Users/efulop/codechecker/venv/lib/python3.14/site-packages/codechecker_server/cli/server.py", line 401, in __handle
return main(args)
File "/Users/efulop/codechecker/venv/lib/python3.14/site-packages/codechecker_server/cli/server.py", line 1087, in main
return server_init_start(args)
File "/Users/efulop/codechecker/venv/lib/python3.14/site-packages/codechecker_server/cli/server.py", line 1036, in server_init_start
return server.start_server(args.config_directory,
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
args.workspace,
^^^^^^^^^^^^^^^
...<7 lines>...
environ,
^^^^^^^^
machine_id)
^^^^^^^^^^^
File "/Users/efulop/codechecker/venv/lib/python3.14/site-packages/codechecker_server/server.py", line 1049, in start_server
all_success, fails = _do_db_cleanups(config_sql_server,
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
context,
^^^^^^^^
check_env)
^^^^^^^^^^
File "/Users/efulop/codechecker/venv/lib/python3.14/site-packages/codechecker_server/server.py", line 602, in _do_db_cleanups
with Pool(max_workers=thr_count) as executor:
~~~~^^^^^^^^^^^^^^^^^^^^^^^
TypeError: BaseContext.Pool() got an unexpected keyword argument 'max_workers'
Environment:
- macOS (darwin)
- Python 3.14.2
- CodeChecker server startup
Proposed Solution
Add a compatibility wrapper class that:
- Accepts
max_workersparameter (converted toprocessesformultiprocess.Pool) - Supports context manager protocol (
withstatement) - Implements
map()with multiple iterables support (matchingProcessPoolExecutor.map()) - Provides
close()andjoin()methods
This ensures consistent API across platforms without requiring platform-specific conditionals in user code.
Affected Files
codechecker_common/compatibility/multiprocessing.py- Any code using
Pool(max_workers=...)from the compatibility module