Skip to content

Commit 9916e6f

Browse files
committed
update docs
Signed-off-by: yiliu30 <[email protected]>
1 parent b916195 commit 9916e6f

File tree

1 file changed

+18
-1
lines changed

1 file changed

+18
-1
lines changed

scripts/inc_woq_g2_bkc.md

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,8 +38,25 @@ This script 1) converts official model weights from `torch.float8_e4m3fn` format
3838
>
3939
>For more details, refer to the `INC WOQ ReQuant` section in the `single_16k_len_inc.sh` script.
4040
41+
42+
43+
### Configure the Measurement Statistics Results
44+
45+
The environment variable `INC_MEASUREMENT_DUMP_PATH_PREFIX` specifies the root directory where measurement statistics were saved.
46+
The final path is constructed by joining this root directory with the `dump_stats_path` defined in the quantization JSON file specified by the `QUANT_CONFIG` environment variable.
47+
48+
Example:
49+
```bash
50+
INC_MEASUREMENT_DUMP_PATH_PREFIX=/mnt/disk3/vllm-fork
51+
dump_stats_path (from config): "scripts/nc_workspace_measure_kvache/inc_measure_output"
52+
Resulting full path: "/mnt/disk3/vllm-fork/scripts/nc_workspace_measure_kvache/inc_measure_output_xx"
53+
```
54+
4155
> [!CAUTION]
42-
> Before running the benchmark, make sure to update the `model_path` in the `single_16k_len_inc.sh` script.
56+
> Before running the benchmark, update the following variables in the single_16k_len_inc.sh script:
57+
> - `model_path`
58+
> - `QUANT_CONFIG`
59+
> - `INC_MEASUREMENT_DUMP_PATH_PREFIX`
4360
4461
### 3.1 BF16 KV + Per-Channel Quantization
4562

0 commit comments

Comments
 (0)