Skip to content

Commit f76cc10

Browse files
zhuzilinzhuzilin
committed
deploy: 39a8c2f 
1 parent 6ce7e4e commit f76cc10

File tree

12 files changed

+30
-62
lines changed

12 files changed

+30
-62
lines changed

‎.doctrees/_examples_synced/low_precision/README.doctree‎

-944 Bytes
Binary file not shown.

‎.doctrees/environment.pickle‎

-777 Bytes
Binary file not shown.

‎_examples_synced/low_precision/README.html‎

Lines changed: 9 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -448,9 +448,8 @@ <h2> Contents </h2>
448448
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#int4-training-examples">INT4 Training Examples</a><ul class="visible nav section-nav flex-column">
449449
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#id1">Files</a></li>
450450
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#id2">Quick Start</a><ul class="nav section-nav flex-column">
451-
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#configure-training-arguments">1. Configure Training Arguments</a></li>
452-
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#convert-huggingface-weights-to-int4">2. Convert HuggingFace Weights to INT4</a></li>
453-
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#start-int4-training">3. Start INT4 Training</a></li>
451+
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#convert-huggingface-weights-to-int4">1. Convert HuggingFace Weights to INT4</a></li>
452+
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#start-int4-training">2. Start INT4 Training</a></li>
454453
</ul>
455454
</li>
456455
</ul>
@@ -546,27 +545,20 @@ <h3>Files<a class="headerlink" href="#id1" title="Link to this heading">#</a></h
546545
</section>
547546
<section id="id2">
548547
<h3>Quick Start<a class="headerlink" href="#id2" title="Link to this heading">#</a></h3>
549-
<section id="configure-training-arguments">
550-
<h4>1. Configure Training Arguments<a class="headerlink" href="#configure-training-arguments" title="Link to this heading">#</a></h4>
551-
<p>Ensure your training script is properly configured. For training tasks, you must add the following flag to your launch arguments:</p>
552-
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>--int4-params-rollout
553-
</pre></div>
554-
</div>
555-
</section>
556548
<section id="convert-huggingface-weights-to-int4">
557-
<h4>2. Convert HuggingFace Weights to INT4<a class="headerlink" href="#convert-huggingface-weights-to-int4" title="Link to this heading">#</a></h4>
549+
<h4>1. Convert HuggingFace Weights to INT4<a class="headerlink" href="#convert-huggingface-weights-to-int4" title="Link to this heading">#</a></h4>
558550
<p>First, download the PTQ (Post-Training Quantization) calibration dataset from HuggingFace:
559551
<a class="reference external" href="https://huggingface.co/datasets/Salesforce/wikitext/tree/main/wikitext-2-raw-v1">https://huggingface.co/datasets/Salesforce/wikitext/tree/main/wikitext-2-raw-v1</a></p>
560552
<p>Next, use the <code class="docutils literal notranslate"><span class="pre">tools/convert_hf_to_hf_int4.py</span></code> script to convert BF16 weights to INT4 format. Ensure that the <code class="docutils literal notranslate"><span class="pre">--hf-checkpoint</span></code> parameter points to a directory where <code class="docutils literal notranslate"><span class="pre">config.json</span></code> contains the correct <code class="docutils literal notranslate"><span class="pre">quantization_config</span></code>. Slime will automatically utilize INT4 quantization during weight updates.</p>
561553
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python<span class="w"> </span>tools/convert_hf_to_hf_int4.py<span class="w"> </span><span class="se">\</span>
562-
<span class="w"> </span>--model_id<span class="w"> </span>/path/to/your/original/models<span class="w"> </span><span class="se">\</span>
563-
<span class="w"> </span>--output_dir<span class="w"> </span>/path/to/your/save/models<span class="w"> </span><span class="se">\</span>
564-
<span class="w"> </span>--local_data_path<span class="w"> </span>/path/to/your/wikitext
554+
<span class="w"> </span>--input-dir<span class="w"> </span>/path/to/your/original/models<span class="w"> </span><span class="se">\</span>
555+
<span class="w"> </span>--output-dir<span class="w"> </span>/path/to/your/save/models<span class="w"> </span><span class="se">\</span>
556+
<span class="w"> </span>--data-dir<span class="w"> </span>/path/to/your/wikitext
565557
</pre></div>
566558
</div>
567559
</section>
568560
<section id="start-int4-training">
569-
<h4>3. Start INT4 Training<a class="headerlink" href="#start-int4-training" title="Link to this heading">#</a></h4>
561+
<h4>2. Start INT4 Training<a class="headerlink" href="#start-int4-training" title="Link to this heading">#</a></h4>
570562
<p>You need to configure the specific environment variables for quantization settings.</p>
571563
<p><strong>Environment Variables:</strong></p>
572564
<ul class="simple">
@@ -644,9 +636,8 @@ <h4>3. Start INT4 Training<a class="headerlink" href="#start-int4-training" titl
644636
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#int4-training-examples">INT4 Training Examples</a><ul class="visible nav section-nav flex-column">
645637
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#id1">Files</a></li>
646638
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#id2">Quick Start</a><ul class="nav section-nav flex-column">
647-
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#configure-training-arguments">1. Configure Training Arguments</a></li>
648-
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#convert-huggingface-weights-to-int4">2. Convert HuggingFace Weights to INT4</a></li>
649-
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#start-int4-training">3. Start INT4 Training</a></li>
639+
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#convert-huggingface-weights-to-int4">1. Convert HuggingFace Weights to INT4</a></li>
640+
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#start-int4-training">2. Start INT4 Training</a></li>
650641
</ul>
651642
</li>
652643
</ul>

‎_sources/_examples_synced/low_precision/README.md‎

Lines changed: 5 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -85,27 +85,20 @@ This guide provides examples for INT4 STE (Straight-Through Estimator) training
8585

8686
### Quick Start
8787

88-
#### 1. Configure Training Arguments
89-
Ensure your training script is properly configured. For training tasks, you must add the following flag to your launch arguments:
90-
91-
```bash
92-
--int4-params-rollout
93-
```
94-
95-
#### 2. Convert HuggingFace Weights to INT4
88+
#### 1. Convert HuggingFace Weights to INT4
9689
First, download the PTQ (Post-Training Quantization) calibration dataset from HuggingFace:
9790
[https://huggingface.co/datasets/Salesforce/wikitext/tree/main/wikitext-2-raw-v1](https://huggingface.co/datasets/Salesforce/wikitext/tree/main/wikitext-2-raw-v1)
9891

9992
Next, use the `tools/convert_hf_to_hf_int4.py` script to convert BF16 weights to INT4 format. Ensure that the `--hf-checkpoint` parameter points to a directory where `config.json` contains the correct `quantization_config`. Slime will automatically utilize INT4 quantization during weight updates.
10093

10194
```bash
10295
python tools/convert_hf_to_hf_int4.py \
103-
--model_id /path/to/your/original/models \
104-
--output_dir /path/to/your/save/models \
105-
--local_data_path /path/to/your/wikitext
96+
--input-dir /path/to/your/original/models \
97+
--output-dir /path/to/your/save/models \
98+
--data-dir /path/to/your/wikitext
10699
```
107100

108-
#### 3. Start INT4 Training
101+
#### 2. Start INT4 Training
109102

110103
You need to configure the specific environment variables for quantization settings.
111104

‎objects.inv‎

-54 Bytes
Binary file not shown.

‎searchindex.js‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎zh/.doctrees/_examples_synced/low_precision/README.doctree‎

-948 Bytes
Binary file not shown.

‎zh/.doctrees/environment.pickle‎

-777 Bytes
Binary file not shown.

‎zh/_examples_synced/low_precision/README.html‎

Lines changed: 9 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -444,9 +444,8 @@ <h2> 目录 </h2>
444444
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#int4-training-examples">INT4 Training Examples</a><ul class="visible nav section-nav flex-column">
445445
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#id1">Files</a></li>
446446
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#id2">Quick Start</a><ul class="nav section-nav flex-column">
447-
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#configure-training-arguments">1. Configure Training Arguments</a></li>
448-
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#convert-huggingface-weights-to-int4">2. Convert HuggingFace Weights to INT4</a></li>
449-
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#start-int4-training">3. Start INT4 Training</a></li>
447+
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#convert-huggingface-weights-to-int4">1. Convert HuggingFace Weights to INT4</a></li>
448+
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#start-int4-training">2. Start INT4 Training</a></li>
450449
</ul>
451450
</li>
452451
</ul>
@@ -542,27 +541,20 @@ <h3>Files<a class="headerlink" href="#id1" title="Link to this heading">#</a></h
542541
</section>
543542
<section id="id2">
544543
<h3>Quick Start<a class="headerlink" href="#id2" title="Link to this heading">#</a></h3>
545-
<section id="configure-training-arguments">
546-
<h4>1. Configure Training Arguments<a class="headerlink" href="#configure-training-arguments" title="Link to this heading">#</a></h4>
547-
<p>Ensure your training script is properly configured. For training tasks, you must add the following flag to your launch arguments:</p>
548-
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>--int4-params-rollout
549-
</pre></div>
550-
</div>
551-
</section>
552544
<section id="convert-huggingface-weights-to-int4">
553-
<h4>2. Convert HuggingFace Weights to INT4<a class="headerlink" href="#convert-huggingface-weights-to-int4" title="Link to this heading">#</a></h4>
545+
<h4>1. Convert HuggingFace Weights to INT4<a class="headerlink" href="#convert-huggingface-weights-to-int4" title="Link to this heading">#</a></h4>
554546
<p>First, download the PTQ (Post-Training Quantization) calibration dataset from HuggingFace:
555547
<a class="reference external" href="https://huggingface.co/datasets/Salesforce/wikitext/tree/main/wikitext-2-raw-v1">https://huggingface.co/datasets/Salesforce/wikitext/tree/main/wikitext-2-raw-v1</a></p>
556548
<p>Next, use the <code class="docutils literal notranslate"><span class="pre">tools/convert_hf_to_hf_int4.py</span></code> script to convert BF16 weights to INT4 format. Ensure that the <code class="docutils literal notranslate"><span class="pre">--hf-checkpoint</span></code> parameter points to a directory where <code class="docutils literal notranslate"><span class="pre">config.json</span></code> contains the correct <code class="docutils literal notranslate"><span class="pre">quantization_config</span></code>. Slime will automatically utilize INT4 quantization during weight updates.</p>
557549
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python<span class="w"> </span>tools/convert_hf_to_hf_int4.py<span class="w"> </span><span class="se">\</span>
558-
<span class="w"> </span>--model_id<span class="w"> </span>/path/to/your/original/models<span class="w"> </span><span class="se">\</span>
559-
<span class="w"> </span>--output_dir<span class="w"> </span>/path/to/your/save/models<span class="w"> </span><span class="se">\</span>
560-
<span class="w"> </span>--local_data_path<span class="w"> </span>/path/to/your/wikitext
550+
<span class="w"> </span>--input-dir<span class="w"> </span>/path/to/your/original/models<span class="w"> </span><span class="se">\</span>
551+
<span class="w"> </span>--output-dir<span class="w"> </span>/path/to/your/save/models<span class="w"> </span><span class="se">\</span>
552+
<span class="w"> </span>--data-dir<span class="w"> </span>/path/to/your/wikitext
561553
</pre></div>
562554
</div>
563555
</section>
564556
<section id="start-int4-training">
565-
<h4>3. Start INT4 Training<a class="headerlink" href="#start-int4-training" title="Link to this heading">#</a></h4>
557+
<h4>2. Start INT4 Training<a class="headerlink" href="#start-int4-training" title="Link to this heading">#</a></h4>
566558
<p>You need to configure the specific environment variables for quantization settings.</p>
567559
<p><strong>Environment Variables:</strong></p>
568560
<ul class="simple">
@@ -640,9 +632,8 @@ <h4>3. Start INT4 Training<a class="headerlink" href="#start-int4-training" titl
640632
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#int4-training-examples">INT4 Training Examples</a><ul class="visible nav section-nav flex-column">
641633
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#id1">Files</a></li>
642634
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#id2">Quick Start</a><ul class="nav section-nav flex-column">
643-
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#configure-training-arguments">1. Configure Training Arguments</a></li>
644-
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#convert-huggingface-weights-to-int4">2. Convert HuggingFace Weights to INT4</a></li>
645-
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#start-int4-training">3. Start INT4 Training</a></li>
635+
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#convert-huggingface-weights-to-int4">1. Convert HuggingFace Weights to INT4</a></li>
636+
<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#start-int4-training">2. Start INT4 Training</a></li>
646637
</ul>
647638
</li>
648639
</ul>

‎zh/_sources/_examples_synced/low_precision/README.md‎

Lines changed: 5 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -85,27 +85,20 @@ This guide provides examples for INT4 STE (Straight-Through Estimator) training
8585

8686
### Quick Start
8787

88-
#### 1. Configure Training Arguments
89-
Ensure your training script is properly configured. For training tasks, you must add the following flag to your launch arguments:
90-
91-
```bash
92-
--int4-params-rollout
93-
```
94-
95-
#### 2. Convert HuggingFace Weights to INT4
88+
#### 1. Convert HuggingFace Weights to INT4
9689
First, download the PTQ (Post-Training Quantization) calibration dataset from HuggingFace:
9790
[https://huggingface.co/datasets/Salesforce/wikitext/tree/main/wikitext-2-raw-v1](https://huggingface.co/datasets/Salesforce/wikitext/tree/main/wikitext-2-raw-v1)
9891

9992
Next, use the `tools/convert_hf_to_hf_int4.py` script to convert BF16 weights to INT4 format. Ensure that the `--hf-checkpoint` parameter points to a directory where `config.json` contains the correct `quantization_config`. Slime will automatically utilize INT4 quantization during weight updates.
10093

10194
```bash
10295
python tools/convert_hf_to_hf_int4.py \
103-
--model_id /path/to/your/original/models \
104-
--output_dir /path/to/your/save/models \
105-
--local_data_path /path/to/your/wikitext
96+
--input-dir /path/to/your/original/models \
97+
--output-dir /path/to/your/save/models \
98+
--data-dir /path/to/your/wikitext
10699
```
107100

108-
#### 3. Start INT4 Training
101+
#### 2. Start INT4 Training
109102

110103
You need to configure the specific environment variables for quantization settings.
111104

0 commit comments

Comments
 (0)