v0.1.5 Released!

[ver217]

Main Features

1. Enhance ColoTensor. We can train BERT (from hugging face) without modifying model now.

What's Changed

ColoTensor

• [Tensor] add ColoTensor TP1Dcol Embedding by @Wesley-Jzy in [Tensor] add ColoTensor TP1Dcol Embedding #899
• [Tensor] add embedding tp1d row by @Wesley-Jzy in [Tensor] add embedding tp1d row #904
• [Tensor] update pytest.mark.parametrize in tensor tests by @Wesley-Jzy in [Tensor] update pytest.mark.parametrize in tensor tests #913
• [Tensor] init ColoParameter by @feifeibear in [Tensor] init ColoParameter #914
• [Tensor] add a basic bert. by @Wesley-Jzy in [Tensor] add a basic bert. #911
• [Tensor] polish model test by @feifeibear in [Tensor] polish model test #915
• [Tensor] fix test_model by @Wesley-Jzy in [Tensor] fix test_model #916
• [Tensor] add 1d vocab loss by @Wesley-Jzy in [Tensor] add 1d vocab loss #918
• [Graph] building computing graph with ColoTensor, Linear only by @feifeibear in [Graph] building computing graph with ColoTensor, Linear only #917
• [Tensor] add from_pretrained support and bert pretrained test by @Wesley-Jzy in [Tensor] add from_pretrained support and bert pretrained test #921
• [Tensor] test pretrain loading on multi-process by @feifeibear in [Tensor] test pretrain loading on multi-process #922
• [tensor] hijack addmm for colo tensor by @ver217 in [tensor] hijack addmm for colo tensor #923
• [tensor] colo tensor overrides mul by @ver217 in [tensor] colo tensor overrides mul #927
• [Tensor] simplify named param by @Wesley-Jzy in [Tensor] simplify named param #928
• [Tensor] fix init context by @Wesley-Jzy in [Tensor] fix init context #931
• [Tensor] add optimizer to bert test by @Wesley-Jzy in [Tensor] add optimizer to bert test #933
• [tensor] design DistSpec and DistSpecManager for ColoTensor by @ver217 in [tensor] design DistSpec and DistSpecManager for ColoTensor #934
• [Tensor] add DistSpec for loss and test_model by @Wesley-Jzy in [Tensor] add DistSpec for loss and test_model #947
• [tensor] derive compute pattern from dist spec by @ver217 in [tensor] derive compute pattern from dist spec #971

Pipeline Parallelism

• [pipelinable]use pipelinable to support GPT model. by @YuliangLiu0306 in [pipelinable]use pipelinable to support GPT model. #903

CI

• [CI] add CI for releasing bdist wheel by @ver217 in [CI] add CI for releasing bdist wheel #901
• [CI] fix release bdist CI by @ver217 in [CI] fix release bdist CI #902
• [ci] added wheel build scripts by @FrankLeeeee in [ci] added wheel build scripts #910

Misc

• [Bot] Synchronize Submodule References by @github-actions in [Bot] Synchronize Submodule References #907
• [setup] update local version format by @ver217 in [setup] update local version format #909
• [Bot] Synchronize Submodule References by @github-actions in [Bot] Synchronize Submodule References #912
• [setup] update cuda ext cc flags by @ver217 in [setup] update cuda ext cc flags #919
• [setup] support more cuda architectures by @ver217 in [setup] support more cuda architectures #920
• [Bot] Synchronize Submodule References by @github-actions in [Bot] Synchronize Submodule References #929
• [Bot] Synchronize Submodule References by @github-actions in [Bot] Synchronize Submodule References #932
• [Bot] Synchronize Submodule References by @github-actions in [Bot] Synchronize Submodule References #960
• [NFC] update results on a single GPU, highlight quick view by @binmakeswell in [NFC] update results on a single GPU, highlight quick view #981

Full Changelog: v0.1.4...v0.1.5

---

This discussion was created from the release v0.1.5 Released! .