ExaServe is being generalized from an Aurora/Intel-XPU/PBS-specific stack into a portable HPC serving platform: Ray + engine-of-choice + proxy-of-choice, with pluggable schedulers and vendors. This directory holds the design; the table below is the source of truth for what is built vs. proposed.
| Step | Scope | Doc | Status |
|---|---|---|---|
| (1) | Rename aurora_rayserver → exaserve (package, CLI, env vars, brand) |
— (commit 6a2faa9) |
Done & validated on real compute (1-node all-features, 2-node 24-replica proxy/internode; 405B PP re-smoke in progress) |
| (2) | Pluggable engine interface (+ proxy polish) | pluggable_interfaces.md | Partially built: engines/ ABC + NullEngine + registry shipped; server.py extraction (VLLM/SGLang engines behind one EngineWorker) not started |
| (3) | Pluggable scheduler (PBS → Slurm) | scheduler_abstraction.md | Design only |
| (4) | Pluggable vendor (XPU → CUDA/ROCm) + site config | vendor_site_abstraction.md | Design only |
| (5) | Update doc/exaserve.md, README, paper to the generic framing |
— | Not started (do last, once the above land) |
All four axes converge on one shape, already proven by the proxy layer
(proxy/base.py::ProxyBackend + proxy/get_proxy()):
an ABC + a lazy registry + a single host/caller that touches only the interface, selected by an
EXASERVE_*env var or the site config.
- Proxy:
ProxyBackend/get_proxy— already clean (the template). - Engine:
EngineBackend/get_engine— interface shipped, host wiring pending. - Scheduler:
SchedulerBackend/get_scheduler— proposed. - Vendor:
VendorBackend/get_vendor— proposed. - Site:
SiteConfigcomposes the above (picks vendor + scheduler, supplies facts).
Any refactor of the serving/launch path must re-pass the smoke set the rename was
validated against, with numbers matching baselines:
nullcompute_smoke_1node, refcard_smoke_1node (≈28.6 rps, 0 err),
smoke_slo_stream_{proxy,direct,rayserve}, serverstats_ttft_{off,on}delay_1node,
serverstats_ttft_ondelay_2node (24 replicas, ≈39 rps, 0 err), and the 405B PP
runs (pp405b_pp2_proxyfix 4-node, pp405b_verify_2node).
Aurora subjob/keepalive dev tooling, env_aurora//opt/aurora/frameworks
module, the Intel-XPU EXASERVE_XPU_* workarounds, and the PVC/Triton-SYCL
quirks — these are inputs to the abstractions, consumed via SiteConfig /
VendorBackend, never hardcoded in the serving core.
Slurm end-to-end and NVIDIA/AMD end-to-end cannot be validated on Aurora (PBS + XPU only). Those backends can be unit-tested for command/env shape here; real validation is owed on a Slurm system and CUDA/ROCm hardware respectively, and must be flagged unproven until then.