Skip to content

Fixes #220: Change worker exception to BaseException#221

Open
dxdc wants to merge 2 commits intouqfoundation:masterfrom
dxdc:patch-1
Open

Fixes #220: Change worker exception to BaseException#221
dxdc wants to merge 2 commits intouqfoundation:masterfrom
dxdc:patch-1

Conversation

@dxdc
Copy link

@dxdc dxdc commented May 5, 2025

Summary

See: #220

There may be more work required as I'm not sure how extensively these changes need to be propagated. I'm happy to add tests, etc. but I am not very familiar with the project structure and it seems the code is duplicated into multiple places.

Checklist

Documentation and Tests

  • Added relevant tests that run with python tests/__main__.py, and pass.
  • Added relevant documentation that builds in sphinx without error.
  • Added new features that are documented with examples.
  • Artifacts produced with the main branch work as expected under this PR.

Release Management

  • Added "Fixes #NNN" in the PR body, referencing the issue (#NNN) it closes.
  • Added a comment to issue #NNN, linking back to this PR.
  • Added rationale for any breakage of backwards compatibility.
  • Requested a review.

@mmckerns
Copy link
Member

mmckerns commented May 7, 2025

The code mirrors the stdlib multiprocessing, so there's a "directory" for each major version of python (and also pypy), with a fork of multiprocessing in each from the corresponding version. You edited py3.11, which is the fork for python 3.11. So, often an edit needs to be duplicated across a few versions of python (i.e. directories). Since the code is forked, the tests are also forked... so for python 3.11, the tests are at py3.11/multiprocess/tests

@dxdc
Copy link
Author

dxdc commented May 7, 2025

@mmckerns ok thanks. I'm not sure how to get the tests to pass, as I'm not very familiar with this library. but these tests definitely seem to hang.

    def test_map_sigterm_base_exception(self):
        def _sigterm_worker(x):
            def _handler(signum, frame):
                raise SystemExit(f"worker received SIGTERM on input={x}")
        
            signal.signal(signal.SIGTERM, _handler)
            os.kill(os.getpid(), signal.SIGTERM)

        try:
            self.pool.map_async(_sigterm_worker, list(range(3))).get(timeout=SHUTDOWN_TIMEOUT)
        except multiprocessing.TimeoutError:
            self.fail("pool.map_async with sigterm stalled on base exception")

    def test_imap_sigterm_base_exception(self):
        def _sigterm_worker(x):
            def _handler(signum, frame):
                raise SystemExit(f"worker received SIGTERM on input={x}")
        
            signal.signal(signal.SIGTERM, _handler)
            os.kill(os.getpid(), signal.SIGTERM)

        try:
            for _ in self.pool.imap(_sigterm_worker, list(range(3))):
                pass
        except multiprocessing.TimeoutError:
            self.fail("pool.map_async with sigterm stalled on base exception")




@dxdc
Copy link
Author

dxdc commented Aug 15, 2025

@mmckerns any chance you could review this again? I made the other changes

@dxdc
Copy link
Author

dxdc commented Dec 21, 2025

@mmckerns just bumping this in case it got buried — whenever you have a chance, I’d appreciate another look. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments