Can I add Qwen Instruct models?

[ITHwang]

Hi, I'm a big fan of candle so I implemented the LLMs below in my repo forked from candle:

• Qwen2-Instruct models including quantized version
• Qwen2.5-Instruct models including quantized version

Furthermore, I published Qwen2.5 Instruct demo in my huggingface space.

Can I add these qwen models into candle-examples and candle-wasm-examples?

[LaurentMazare]

Sure, feel free to make a PR to add these!

[lucasjinreal]

Hello, does the quantized version as quick as llama.cpp on macos?

[ITHwang]

@lucasjinreal
For sure, the quantized version is in the #2797

[MonolithFoundation]

Hello, when using instruct model (no quantize), in chatting mode, when pipe runs twice, there will be error:

called `Result::unwrap()` on an `Err` value: shape mismatch in broadcast_add, lhs: [1, 14, 5, 243], rhs: [1, 1, 5, 5]

Anybody would give some hints? It's so weired that it even can not runs multiple times.

[ITHwang]

Could you provide the part of your command or the whole error log for reproducibility?