Skip to content

feat: add vllm rust frontend#457

Merged
gufengc merged 8 commits into
mainfrom
codex/vllm-rust-frontend
May 28, 2026
Merged

feat: add vllm rust frontend#457
gufengc merged 8 commits into
mainfrom
codex/vllm-rust-frontend

Conversation

@gufengc
Copy link
Copy Markdown
Collaborator

@gufengc gufengc commented May 27, 2026

Summary

Adds vLLM Rust frontend support as Parallax's HTTP entrypoint and wires executors to the vLLM engine-core MessagePack/ZMQ protocol.

What changed

  • Launches vllm-rs frontend from parallax.launch and adds a build helper for installing the Rust frontend binary.
  • Adds engine-core protocol helpers for request decoding, sampling parameter mapping, routing metadata extraction, ready registration, token outputs, terminal outputs, aborts, and errors.
  • Updates MLX, SGLang, and vLLM executors to emit vLLM engine-core outputs instead of the old Python HTTP-server payloads.
  • Forwards Parallax routing metadata through vllm_xargs and updates frontend streaming handling for reasoning deltas.
  • Refreshes frontend dist assets and replaces old HTTP handler tests with backend request and engine-core protocol coverage.

Impact

Parallax now relies on the official vLLM Rust frontend for OpenAI-compatible HTTP serving. Backend requests carry scheduler/routing metadata through supported vLLM extension arguments, while executor outputs use vLLM-compatible finish reasons and finished-request lists.

Validation

  • uv run --no-sync --with pre-commit pre-commit run --all-files
  • uv run --no-sync --with pytest --with pytest-mock --with pytest-cov pytest

@gufengc gufengc changed the title [codex] add vLLM Rust frontend feat: add vLLM Rust frontend May 27, 2026
@gufengc gufengc changed the title feat: add vLLM Rust frontend feat: add vllm rust frontend May 27, 2026
@gufengc gufengc marked this pull request as ready for review May 28, 2026 08:43
@gufengc gufengc requested a review from a team May 28, 2026 08:43
@gufengc gufengc merged commit 328c99f into main May 28, 2026
11 checks passed
@gufengc gufengc deleted the codex/vllm-rust-frontend branch May 28, 2026 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant