Labels: enhancement
Context
dotflow/core/workflow.py:20-31:
if sys.platform == "win32":
_mp = get_context("spawn")
else:
if sys.platform == "darwin":
os.environ.setdefault("OBJC_DISABLE_INITIALIZE_FORK_SAFETY", "YES")
_mp = get_context("fork")
Problems:
- macOS
fork with the Objective-C runtime is a known deadlock
source. OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES only suppresses the warning; native libraries (numpy, cryptography, grpcio) can still crash silently.
- Python 3.14 changes the macOS default to
spawn for this reason.
The library will inherit that behavior whether we like it or not; shipping ahead of the change avoids surprise breakage.
fork inherits all open file descriptors from the parent, which
causes subtle bugs with database connection pools (the same socket is shared between processes).
Concept
Default mp_context to spawn on every platform. Allow override via env var or Config.
Proposed change
import os
DEFAULT_MP_CONTEXT = os.getenv("DOTFLOW_MP_CONTEXT", "spawn")
_mp = get_context(DEFAULT_MP_CONTEXT)
Optionally exposed on Config:
config = Config(mp_context="fork") # opt back in if you accept the risk
Migration notes
spawn re-imports the worker module on each process start. Workflows that rely on importable steps will work; workflows that pass non-picklable closures will fail. Document the requirement.
Acceptance criteria
Labels:
enhancementContext
dotflow/core/workflow.py:20-31:Problems:
forkwith the Objective-C runtime is a known deadlocksource.
OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YESonly suppresses the warning; native libraries (numpy, cryptography, grpcio) can still crash silently.spawnfor this reason.The library will inherit that behavior whether we like it or not; shipping ahead of the change avoids surprise breakage.
forkinherits all open file descriptors from the parent, whichcauses subtle bugs with database connection pools (the same socket is shared between processes).
Concept
Default
mp_contexttospawnon every platform. Allow override via env var orConfig.Proposed change
Optionally exposed on
Config:Migration notes
spawnre-imports the worker module on each process start. Workflows that rely on importable steps will work; workflows that pass non-picklable closures will fail. Document the requirement.Acceptance criteria
spawnon darwin and linuxDOTFLOW_MP_CONTEXTenv var accepted (spawn|fork|forkserver)