Skip to content

macOS/Windows incompatibility: max_workers parameter not supported by multiprocess.Pool #4733

@gamesh411

Description

@gamesh411

macOS/Windows incompatibility: max_workers parameter not supported by multiprocess.Pool

Problem

Code using the max_workers parameter with the multiprocessing compatibility layer fails on macOS and Windows platforms. The multiprocess.Pool (used on darwin/win32) expects a processes parameter instead of max_workers, which breaks cross-platform compatibility.

Root Cause

The codechecker_common.compatibility.multiprocessing module currently exports multiprocess.Pool directly on macOS/Windows platforms, which has a different API than ProcessPoolExecutor used on Linux:

  • macOS/Windows: multiprocess.Pool(processes=N)
  • Linux: ProcessPoolExecutor(max_workers=N)

This API mismatch prevents code from using max_workers consistently across all platforms.

Background

The multiprocess package is used on macOS/Windows instead of the standard library multiprocessing due to reliability issues (see commits 803e446, d56e216):

  • On macOS: Standard multiprocessing has known issues
  • On Windows: ProcessPoolExecutor fails due to fork() limitations and PYTHONPATH issues
  • On Linux: ProcessPoolExecutor works reliably

Expected Behavior

Code should be able to use max_workers parameter consistently across all platforms:

from codechecker_common.compatibility.multiprocessing import Pool

with Pool(max_workers=4) as pool:
    results = pool.map(func, items)

Actual Behavior

On macOS/Windows, using max_workers fails because multiprocess.Pool doesn't accept this parameter:

Traceback (most recent call last):
  File "/Users/efulop/codechecker/venv/lib/python3.14/site-packages/codechecker_common/cli.py", line 259, in main
    sys.exit(args.func(args))
             ~~~~~~~~~^^^^^^
  File "/Users/efulop/codechecker/venv/lib/python3.14/site-packages/codechecker_server/cli/server.py", line 401, in __handle
    return main(args)
  File "/Users/efulop/codechecker/venv/lib/python3.14/site-packages/codechecker_server/cli/server.py", line 1087, in main
    return server_init_start(args)
  File "/Users/efulop/codechecker/venv/lib/python3.14/site-packages/codechecker_server/cli/server.py", line 1036, in server_init_start
    return server.start_server(args.config_directory,
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
                               args.workspace,
                               ^^^^^^^^^^^^^^^
    ...<7 lines>...
                               environ,
                               ^^^^^^^^
                               machine_id)
                               ^^^^^^^^^^^
  File "/Users/efulop/codechecker/venv/lib/python3.14/site-packages/codechecker_server/server.py", line 1049, in start_server
    all_success, fails = _do_db_cleanups(config_sql_server,
                         ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
                                         context,
                                         ^^^^^^^^
                                         check_env)
                                         ^^^^^^^^^^
  File "/Users/efulop/codechecker/venv/lib/python3.14/site-packages/codechecker_server/server.py", line 602, in _do_db_cleanups
    with Pool(max_workers=thr_count) as executor:
         ~~~~^^^^^^^^^^^^^^^^^^^^^^^
TypeError: BaseContext.Pool() got an unexpected keyword argument 'max_workers'

Environment:

  • macOS (darwin)
  • Python 3.14.2
  • CodeChecker server startup

Proposed Solution

Add a compatibility wrapper class that:

  1. Accepts max_workers parameter (converted to processes for multiprocess.Pool)
  2. Supports context manager protocol (with statement)
  3. Implements map() with multiple iterables support (matching ProcessPoolExecutor.map())
  4. Provides close() and join() methods

This ensures consistent API across platforms without requiring platform-specific conditionals in user code.

Affected Files

  • codechecker_common/compatibility/multiprocessing.py
  • Any code using Pool(max_workers=...) from the compatibility module

References

  • Related commits: 803e446, d56e216
  • Issue introduced when multiprocess package was adopted for macOS/Windows support

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions