[V1]: Isolate IPC endpoints per server run#402
Open
Ratish1 wants to merge 4 commits intosgl-project:mainfrom
Open
[V1]: Isolate IPC endpoints per server run#402Ratish1 wants to merge 4 commits intosgl-project:mainfrom
Ratish1 wants to merge 4 commits intosgl-project:mainfrom
Conversation
d074add to
ee450f3
Compare
zhaochenyang20
requested changes
May 6, 2026
ee450f3 to
d35685c
Compare
zhaochenyang20
requested changes
May 6, 2026
Collaborator
|
Please rebase and rerun CI after #403 is merged |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Omni V1 currently derives default IPC endpoints from
endpoints.base_path / config.name. When two V1 servers are launched with the same model name and default IPC settings, they can bind/connect to the same ZMQ socket paths and leak control-plane messages across server instances.This applies the same per-server IPC namespace invariant used for the V0 runtime isolation fix in #263, adapted to the V1 compiler and
StageProcessSpecstartup path.Root Cause
PipelineConfig.namedefaults to the model path. Two Qwen3-Omni V1 servers launched with the same model therefore allocate identical stable endpoints under the same model-name directory.Because V1 uses ZMQ for Coordinator-to-Stage control-plane traffic, sharing these paths can let one server receive another server's stage messages or completions.
Modifications
IpcRuntimeDirlifecycle insglang_omni_v1.config.compiler, matching the V0 per-run IPC directory ownership model from [Server]: Isolate IPC Endpoints Per Server Run #263.tempfile.mkdtemp(...)instead of the stablebase_path / config.namepath.prepare_pipeline_runtime(...)andcompile_pipeline_core(...)so callers that own server lifecycle also own IPC cleanup.MultiProcessPipelineRunnerprepares one endpoint map in the main process and passes it intoStageProcessSpecconstruction, so subprocesses still do not recompile endpoint state.Accuracy Tests
ipc:///tmp/sglang_omni_v1/qwen-qwen3-omni-30b-a3b-instruct-9ta1b5i2/stage_preprocessing.sockipc:///tmp/sglang_omni_v1/qwen-qwen3-omni-30b-a3b-instruct-9ta1b5i2/stage_thinker.sockipc:///tmp/sglang_omni_v1/qwen-qwen3-omni-30b-a3b-instruct-9ta1b5i2/completion.sockipc:///tmp/sglang_omni_v1/qwen-qwen3-omni-30b-a3b-instruct-y3dtxwhb/stage_preprocessing.sockipc:///tmp/sglang_omni_v1/qwen-qwen3-omni-30b-a3b-instruct-1w6lk5as/stage_preprocessing.sockSpeed Tests and Profiling
Not run. This changes server startup endpoint allocation and shutdown cleanup only; it does not touch request scheduling, model execution, relay tensor transfer, or decode hot paths.