Skip to content

v0.1.1

Latest
Latest

Choose a tag to compare

View all tags
@liangyuwang liangyuwang released this 20 Jun 08:28
· 4 commits to main since this release
v0.1.1
3337ad7

We are releasing ZO2 v0.1.1, a minor update to our high-throughput, memory-efficient zeroth-order fine-tuning framework for large language models.

This release improves compatibility with newer models and libraries while retaining the core simplicity and efficiency of ZO2.

🔧 Changes

  • Upgraded transformers dependency to v4.51.3
  • Added support for Qwen/Qwen3, including Qwen3-14B, Qwen3-32B, and other Hugging Face-compatible variants

🚀 Highlights (inherited from v0.1.0)

  • Full-parameter fine-tuning with only 18GB GPU for models up to 175B
  • CPU offloading for gradients and optimizer states
  • Hugging Face Transformers compatibility
  • Minimal codebase for educational and experimental use

🧪 Tested Models

  • OPT-125M to OPT-175B
  • Qwen3-0.6B to Qwen3-32B

📄 Getting Started

git clone https://github.com/liangyuwang/zo2.git
cd zo2/
conda env create -f env.yml
conda activate zo2
Assets 2
Loading