Skip to content

Commit b69e9a7

Browse files
author
amaurya
committed
Add SC'25 MLP-Offload Artifacts
1 parent 7bba400 commit b69e9a7

3 files changed

Lines changed: 19 additions & 0 deletions

File tree

.gitmodules

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
[submodule "mlp-offload/DeepSpeed"]
2+
path = mlp-offload/DeepSpeed
3+
url = https://github.com/DataStates/DeepSpeed.git
4+
branch = dist_nvme_opt
5+
[submodule "mlp-offload/Megatron-DeepSpeed"]
6+
path = mlp-offload/Megatron-DeepSpeed
7+
url = https://github.com/mauryaavinash95/Megatron-DeepSpeed
8+
branch = dist_nvme_opt

MLP-Offload

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Subproject commit e181b47b4b30c0b19e508b27e91a64c07ef9d30a

README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# DataStates Artifacts
2+
This repository hosts the software artifacts developed to ensure reproducibility of the novel designs proposed through the DataStates project.
3+
4+
Artifacts are organized by publication and can be found in the following subfolders:
5+
6+
1. [Network Architecture Search](./nas/): Robert Underwood, Meghana Madhyastha, Randal Burns, and Bogdan Nicolae. 2024. EvoStore: Towards Scalable Storage of Evolving Learning Models. In Proceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing (HPDC '24). Association for Computing Machinery. Pisa, Italy. https://doi.org/10.1145/3625549.3658679.
7+
8+
2. [Deep Optimizer States](./deep-optimizer-states/): Avinash Maurya, Jie Ye, M. Mustafa Rafique, Franck Cappello, and Bogdan Nicolae. 2024. Deep Optimizer States: Towards Scalable Training of Transformer Models using Interleaved Offloading. In Proceedings of the 25th International Middleware Conference (Middleware '24). Association for Computing Machinery. Hong Kong. https://doi.org/10.1145/3652892.3700781
9+
10+
3. [MLP-Offload](./MLP-Offload/): Avinash Maurya, M. Mustafa Rafique, Franck Cappello, and Bogdan Nicolae. 2025. MLP-Offload: Multi-Level, Multi-Path Offloading for LLM Pre-training to Break the GPU Memory Wall. In Proceedings of the International Conference for High-Performance Computing, Networking, Storage, and Analysis (SC '25). Association for Computing Machinery. St. Lous, MO. https://doi.org/10.1145/3712285.3759864.

0 commit comments

Comments
 (0)