Is your feature request related to a problem? Please describe.
GuideLLM defaults to fork as the multiprocessing context for workers. This is considered unsafe for Python applications that contain threads.
Describe the solution you'd like
We should switch to forkserver as the default multiprocessing method; as of Python 3.14 it is the default for MacOS and Linux.
Additional context
There are a few issues that must be addressed before the default can be changed:
- Most multi-processing primitives are created in the default context rather then respecting
GUIDELLM__MP_CONTEXT_TYPE.
- GuideLLM is designed to explicitly import itself and all dependencies on import. This leads to slow process startups on Linux; due mostly to
transformers and torch
Is your feature request related to a problem? Please describe.
GuideLLM defaults to
forkas the multiprocessing context for workers. This is considered unsafe for Python applications that contain threads.Describe the solution you'd like
We should switch to
forkserveras the default multiprocessing method; as of Python 3.14 it is the default for MacOS and Linux.Additional context
There are a few issues that must be addressed before the default can be changed:
GUIDELLM__MP_CONTEXT_TYPE.transformersandtorch