Skip to content

[DeepSeek] Update Docs for Setting Measurement Results Path #1251

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 16, 2025

Conversation

yiliu30
Copy link

@yiliu30 yiliu30 commented May 13, 2025

Depends on intel/neural-compressor#2210

Configure the Measurement Statistics Results

The environment variable INC_MEASUREMENT_DUMP_PATH_PREFIX specifies the root directory where measurement statistics were saved.
The final path is constructed by joining this root directory with the dump_stats_path defined in the quantization JSON file specified by the QUANT_CONFIG environment variable.

Example:

INC_MEASUREMENT_DUMP_PATH_PREFIX=/mnt/disk3/vllm-fork
dump_stats_path (from config): "scripts/nc_workspace_measure_kvache/inc_measure_output"
Resulting full path: "/mnt/disk3/vllm-fork/scripts/nc_workspace_measure_kvache/inc_measure_output_xx"

Caution

Before running the benchmark, update the following variables in the single_16k_len_inc.sh script:

  • model_path
  • QUANT_CONFIG
  • INC_MEASUREMENT_DUMP_PATH_PREFIX

cc @thuang6

Signed-off-by: yiliu30 <[email protected]>
@yiliu30 yiliu30 changed the title [DeepSeek] Update Docs for Setting Measurement Results [DeepSeek] Update Docs for Setting Measurement Results Path May 13, 2025
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR updates the documentation regarding the configuration of measurement statistics results for the quantization workflow. It adds a new section to explain how the full output path is constructed and clarifies the benchmark preparation steps by listing the environment variables that need updating.

  • Added a "Configure the Measurement Statistics Results" section with an example.
  • Updated the benchmark caution message to include additional environment variables.

@yiliu30
Copy link
Author

yiliu30 commented May 13, 2025

@taotod @czhu15 @yangulei Please help review, thx

Signed-off-by: Yi Liu <[email protected]>
@yiliu30 yiliu30 requested a review from czhu15 May 15, 2025 05:21
Copy link

@czhu15 czhu15 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiliu30 yiliu30 merged commit c00a82d into HabanaAI:deepseek_r1 May 16, 2025
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants