New Users: Expert Kit, A Distributed, Expert-Centric Framework for MoE LLM Inference

[Xuanwo]

https://github.com/expert-kit/expert-kit

They are using opendal to save/load tensors from local fs or s3.

---

Expert Kit (EK) is a high-performance framework for scalable MoE (Mixture of Experts) LLM inference. The vision of EK is to provide an efficient foundation of Expert Parallelism (EP) on heterogeneous hardware (e.g., CPU and GPU) over commodity networks (e.g. PCIe, TCP, RDMA), thereby enabling easy deployment and fine-grained expert-level scaling.