Skip to content

[LLM:Bugfix] Fix embedding QNN export paths#4555

Open
ZedingZhang wants to merge 1 commit into
alibaba:masterfrom
ZedingZhang:fix-embedding-qnn-export
Open

[LLM:Bugfix] Fix embedding QNN export paths#4555
ZedingZhang wants to merge 1 commit into
alibaba:masterfrom
ZedingZhang:fix-embedding-qnn-export

Conversation

@ZedingZhang

Copy link
Copy Markdown

Summary

  • detect embedding models in the QNN export wrapper and generate sentence_embeddings IO without logits_index
  • use embedding.mnn / embedding.mnn.weight for embedding model QNN export and config_qnn.json
  • restore embedding exporter output prefix and make generateLlmIO select embedding.mnn with safer qwen3 detection

Fixes #4481

Tests

  • python3 -m py_compile transformers/llm/export/npu/generate_llm_qnn.py transformers/llm/export/llmexport.py
  • inline Python checks for embedding and normal LLM input.json generation
  • cmake -S . -B build -DMNN_BUILD_LLM=ON -DMNN_LLM_BUILD_DEMO=ON -DMNN_BUILD_TEST=OFF -DMNN_BUILD_BENCHMARK=OFF -DMNN_BUILD_DEMO=OFF -DMNN_BUILD_CONVERTER=OFF -DMNN_BUILD_TRAIN=OFF -DMNN_BUILD_QUANTOOLS=OFF
  • cmake --build build --target generateLlmIO -j4

@CLAassistant

CLAassistant commented Jun 19, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@ZedingZhang

Copy link
Copy Markdown
Author

Closing for now to run local Qwen3-Embedding-0.6B export validation before reopening.

@ZedingZhang ZedingZhang reopened this Jun 19, 2026
@ZedingZhang

Copy link
Copy Markdown
Author

Local validation completed for this PR: Qwen3-Embedding-0.6B QNN export passed in a linux/amd64 Docker container with QAIRT/QNN SDK 2.47.0.260601. Verified outputs include qnn/embedding.mnn, embedding.mnn.weight, and 29 graph*.bin context binaries. config_qnn.json points to qnn/embedding.mnn and embedding.mnn.weight with chunk_limits [128,1].

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Qwen3-Embedding-0.6B generate_llm_qnn.py 再最新代码中报错

4 participants