Skip to content

Commit b8e6d3e

Browse files
authored
Update convert to INT4 script path (#1528)
1 parent 6485964 commit b8e6d3e

File tree

1 file changed

+4
-8
lines changed

1 file changed

+4
-8
lines changed

docs/en/advanced/low-precision.md

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -81,16 +81,12 @@ This guide provides examples for INT4 STE (Straight-Through Estimator) training
8181
### Quick Start
8282

8383
1. Convert HuggingFace Weights to INT4
84-
First, download the PTQ (Post-Training Quantization) calibration dataset from HuggingFace:
85-
[https://huggingface.co/datasets/Salesforce/wikitext/tree/main/wikitext-2-raw-v1](https://huggingface.co/datasets/Salesforce/wikitext/tree/main/wikitext-2-raw-v1)
86-
87-
Next, use the `tools/convert_hf_to_int4.py` script to convert BF16 weights to INT4 format. Ensure that the `--hf-checkpoint` parameter points to a directory where `config.json` contains the correct `quantization_config`. slime will automatically utilize INT4 quantization during weight updates.
84+
Use the `tools/convert_hf_to_int4_direct.py` script to convert BF16 weights to INT4 format. Ensure that the `--hf-checkpoint` parameter points to a directory where `config.json` contains the correct `quantization_config`. slime will automatically utilize INT4 quantization during weight updates.
8885

8986
```bash
90-
python tools/convert_hf_to_int4.py \
91-
--input-dir /path/to/your/original/models \
92-
--output-dir /path/to/your/save/models \
93-
--data-dir /path/to/your/wikitext
87+
python tools/convert_hf_to_int4_direct.py \
88+
--model-dir /path/to/your/original/models \
89+
--save-dir /path/to/your/save/models \
9490
```
9591

9692
Note: If you only hope to run with INT4 rollout, you only need to set the `--hf-checkpoint` to the converted INT4 checkpoint.

0 commit comments

Comments
 (0)