Problem
In the LLM QNN export path, transformers/llm/export/npu/generate_llm_qnn.py exposes a --chunk_size argument, but prefill IO generation did not actually honor it.
Before the fix, convert_llm() always called:
makeIOJson(args, 128, hidden_size, mask_type)
which meant the prefill QNN graph could still be generated against 128-token IO even when --chunk_size 256/512 was requested.
Impact
This can make generated QNN artifacts inconsistent with runtime configuration:
- prefill graph shape may not match the requested chunk size
- runtime
chunk_limits and generated graphs may diverge
ppl_eval results can become misleading
- chat / prefill behavior may differ from expectation
Reproduction
Run QNN export with a non-128 chunk size, for example:
python transformers/llm/export/npu/generate_llm_qnn.py \
--model <model_dir> \
--soc_id 43 \
--dsp_arch v73 \
--mnn_path <mnn_build> \
--chunk_size 512
Expected:
- generated prefill IO and QNN graph use 512-token shape
Actual before fix:
- prefill IO generation still used 128
Problem
In the LLM QNN export path,
transformers/llm/export/npu/generate_llm_qnn.pyexposes a--chunk_sizeargument, but prefill IO generation did not actually honor it.Before the fix,
convert_llm()always called:which meant the prefill QNN graph could still be generated against 128-token IO even when
--chunk_size 256/512was requested.Impact
This can make generated QNN artifacts inconsistent with runtime configuration:
chunk_limitsand generated graphs may divergeppl_evalresults can become misleadingReproduction
Run QNN export with a non-128 chunk size, for example:
Expected:
Actual before fix: