Skip to content

Reusable process groups with recovery after crash #8

@cesarpomar

Description

@cesarpomar

Hi, first of all thank you for pytest-isolated.
It fits my use case almost perfectly and solves problems I couldn’t address cleanly with other plugins.

Background / use case

I work with a large collection of tests that may deadlock or crash the process, especially when testing native libraries (C/C++ extensions).

In the past, I implemented a manual solution where tests were executed in subprocesses with timeouts, aborting them safely and propagating exceptions at runtime using tblib.

I evaluated pytest-forked, but its fork-based approach caused issues in my environment, so it wasn’t a good fit.

Your group-based isolation model is exactly what I needed.

Suggestion: Reusable process groups with recovery after crash

Have you considered adding an option where, if a test inside a group crashes the process (e.g. segfault), pytest would:

  1. Mark that test as failed
  2. Recreate the worker process for that group
  3. Continue executing the remaining tests in the same group

Conceptually, something like a “reusable group”

Why this is useful

  • Native libraries can segfault and must be executed in isolated processes to avoid aborting the whole test run.
  • Reusing a process until it crashes is much more efficient than always spawning a new one.
  • This would allow safe and efficient testing of unstable native code while still benefiting from your grouping strategy.

This approach would sit nicely between:

  • Full isolation per test (safe but slower)
  • Single long-lived process (fast but unsafe)

Out of scope (but related)

Another need in my setup is per-test timeouts to handle deadlocks, but I understand this might be outside the scope of pytest-isolated.

Documentation suggestion

It would be great to include an example in the documentation showing how to use pytest-isolated together with coverage, mentioning that:

[tool.coverage.run]
patch = ["subprocess"]

is required to correctly collect coverage from subprocesses.

Thanks again for your work — I’d be happy to help test or discuss the idea further.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions