[Bug] AWQ kernels fail on RTX 2060 (SM75) with dtype mismatch: isTypeMatch<half_t>

[healthyfat]

Checklist

• 1. I have searched for related issues and FAQs (https://github.com/mit-han-lab/nunchaku/blob/main/docs/faq.md) but was unable to find a solution.
• 2. The issue persists in the latest version.
• 3. Please note that without environment information and a minimal reproducible example, it will be difficult for us to reproduce and address the issue, which may delay our response.
• 4. If your report is a question rather than a bug, please submit it as a discussion at https://github.com/mit-han-lab/ComfyUI-nunchaku/discussions/new/choose. Otherwise, this issue will be closed.
• 5. I will do my best to describe the issue in English.

Describe the Bug

ComfyUI crash when loading svdq-int4_r128-qwen-image.

Requested to load QwenImageTEModel_
loaded partially 3641.675 3641.673828125 0
Attempting to release mmap (417)
loaded partially 3641.675 3641.673828125 0
model weight dtype torch.bfloat16, manual cast: torch.float32
model_type FLUX
Requested to load NunchakuQwenImage
loaded partially 3641.6749580383303 3639.3350830078125 0
0%| | 0/10 [00:00<?, ?it/s]

Assertion failed: isTypeMatch<half_t>(_in_feats.dtype()), file C:\Users\muyangl\actions-runner_work\nunchaku\nunchaku\src\kernels\awq\gemv_awq.cu,
line 254

Environment

GPU: NVIDIA GeForce RTX 2060 6GB

Python Version : 3.12.7
Embedded Python: true
Pytorch Version: 2.7.0+cu128

ComfyUI: 0.3.53
Nunchaku version: 1.0.0.dev20250823
ComfyUI-nunchaku version: 1.0.0dev1

model: svdq-int4_r128-qwen-image.safetensors
clip: Qwen2.5-VL-7B-Instruct-Q4_K_M.gguf

Reproduction Steps

Workflow:

[duia]

嗯 我也遇到了一样的问题

[lmxyy]

we haven't tested on the float16 precision. please wait.

[kouziahng]

一样的问题

[lnyuqian]

这个版本支持qwen-image，20系显卡也支持跑图的？我在B站T8大佬那边看到他说不支持20系显卡了？

[sarim]

@lnyuqian I don't think nunchaku qwen supports RTX 20xxx series yet. I also face same error and crash with RTX 2060 RTX.

[lnyuqian]

@sarim 感谢，你们率先尝试。只有等团队开发了

[MaxTan]

@lnyuqian @sarim

comfyui文档上qwen那个版本是支持的，我用RTX 2080TI 22G跑这个模型完全没问题

[liulsg]

@MaxTan If you use GGUF or FP8, it's okay, but using nunchaku is not allowed.

[mikeyoubeach]

@MaxTan如果你使用 GGUF 或 FP8，这是可以的，但使用 nunchaku 是不允许的。

我使用2080ti 22g运行nunchaku_qwen_image模型权重数据类型 torch.bfloat16，手动转换：torch.float32会崩溃，但我强制使用f16精度不崩溃但生成黑图

[lmxyy]

torch.float16 has not been supported yet for qwen-image. Please wait.