Update llama4 example

[mengniwang95]

User description

Type of Change

update example

Description

fix VLLM_FLASHINFER_DISABLE_Q_QUANTIZATION for fp8 kv

---

PR Type

Bug fix

---

Description

• Fix VLLM_FLASHINFER_DISABLE_Q_QUANTIZATION for fp8 kv

---

Diagram Walkthrough

flowchart LR
  A["Set VLLM_FLASHINFER_DISABLE_Q_QUANTIZATION"] -- "from 0 to 1" --> B["For fp8 kv"]

Loading

File Walkthrough

	Relevant files
Bug fix	run_benchmark.sh Update VLLM_FLASHINFER_DISABLE_Q_QUANTIZATION for fp8 kv  examples/pytorch/multimodal-modeling/quantization/auto_round/llama4/run_benchmark.sh Update VLLM_FLASHINFER_DISABLE_Q_QUANTIZATION to 1 for fp8 kv +1/-1

---

[PRAgent4INC]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Configuration Change The change sets VLLM_FLASHINFER_DISABLE_Q_QUANTIZATION to 1 for fp8 kv. Ensure this change aligns with the intended behavior and does not introduce any unintended side effects. export VLLM_FLASHINFER_DISABLE_Q_QUANTIZATION=1

[PRAgent4INC]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Confirm quantization setting correctness Verify that setting VLLM_FLASHINFER_DISABLE_Q_QUANTIZATION to 1 is correct for the intended behavior when kv_cache_dtype is fp8. examples/pytorch/multimodal-modeling/quantization/auto_round/llama4/run_benchmark.sh [55] +export VLLM_FLASHINFER_DISABLE_Q_QUANTIZATION=1 - Suggestion importance[1-10]: 5 __ Why: The suggestion asks to verify the correctness of the quantization setting, which is a reasonable request but does not provide a direct improvement to the code. It should not receive a score above 7.	Low