[Request] SVDQuant int4 weights for Qwen/Qwen-Image-2512 and Qwen/Qwen-Image-Edit-2511

[Milor123]

I would like to request the release of SVDQuant int4 weights for the new Qwen Image models. These base models have been available on Hugging Face since November and December, but there are no Nunchaku-compatible int4 versions available yet.

The specific models are:

1. Qwen/Qwen-Image-2512 (Released in December)
   • HF Link: https://huggingface.co/Qwen/Qwen-Image-2512
2. Qwen/Qwen-Image-Edit-2511 (Released in November)
   • HF Link: https://huggingface.co/Qwen/Qwen-Image-Edit-2511

It would be great to have these quantized to run them efficiently with Nunchaku.

Thank you!

[vgabbo]

+1
I would love that too

[zcfrank1st]

me too

[Tedy50]

Isn't that the pretty much same model as before juts tuned more so how hard it can be to requantize it?

[emogie]

I would to love it too, or better a good description of how to quantize those models by myself ;)

edit: trying it whole time with deepcompressor, but lack of knowledge how to do it

[Maelstrom2014]

yes

[Milor123]

I would love that too, or even better, a good description of how to quantize those models myself ;)

edit: trying it whole time with deepcompressor, but lack of knowledge how to do it

If you figure out how to do it, then remember to share a post with your experience. Thanks !!! Awesome serial !!

[jarkevithwlad]

+1

[neblen]

+1

[emogie]

I'm trying this https://github.com/ussoewwin/deepcompressor-ussoewwin for Qwen-Image

I used the description here:
https://github.com/ussoewwin/deepcompressor-ussoewwin/tree/main/examples/diffusion

I've skipped Step 1 (Evaluation - with a dataset of 5000 Images, it took till I aborted after ~10 hours of run and in image 6, 2 Hours per Image on my PC, for 5000 Images ~10000 hours) and Step 2 (Calibration, I've skipped completly without testing)

I think this both maybe could lead to issues in Quality, but doing now Quantization, can take up to 18 Hours due to the timer.
(main spec Intel 12700 - 128GB - RTX4060 16GB - 2TB M2.SSD)

It seems it will take some time, to verify if it is running as expected.

Most important part is to collect all prerequisits, you'll need to install SDK version for your target Cuda version. I have CUDA 12.8 SDK running.

my current pip list as comparison
please note not all of These packages are necessary to get it to run:

Package Version

---

nunchaku 1.1.0+torch2.9
datasets 3.6.0 <- Downgraded to this Version, newer Version I got an error
setuptools 80.0.0 <- Downgraded to this Version, got a warning for newer version
xformers 0.0.33.post2
python-Levenshtein 0.27.3
Levenshtein 0.27.3
tokenizers 0.21.4
torch 2.9.1+cu128
torchao 0.15.0
torchaudio 2.9.1+cu128
torchmetrics 1.8.2
torchvision 0.24.1+cu128
transformers 4.55.4
triton-windows 3.4.0.post20
psutil 7.1.0
omniconfig 0.1.10
hf_transfer 0.1.9
huggingface-hub 0.35.3
bitsandbytes 0.48.1
flash_attn 2.8.2 <- Self compiled for Windows, Python 312, Torch 2.9.1, cu128

If you want to use same specs you can download flash-attn here:
https://huggingface.co/emogie3D/flash_attention_2.8.2_cu128_torch2.x.x_cp312_win_amd64

other cuda/torch/python versions - following are good sources:
https://huggingface.co/ussoewwin/Flash-Attention-2_for_Windows/tree/main
https://huggingface.co/lldacing/flash-attention-windows-wheel/tree/main
https://github.com/mjun0812/flash-attention-prebuild-wheels/releases?page=2

in root Directory of deepcompressor-ussoewwin i used following command line
python -m deepcompressor.app.diffusion.ptq examples/diffusion/configs/model/qwen-image.yaml examples/diffusion/configs/svdquant/int4.yaml

[vvhitevvizard]

and NVFP4 for 50 series. NVFP4 gets even better results

[emogie]

today morning 01:32 a.m. local time, my quantization was finished, but after this step the script started with an evaluation of the model, time till it will be finnished -> approx 12077 hours

In my opinion totally inefficient. (it seems there is some where an option to switch this option off, but don't know where to set, and event he option is set, I don't know if after a conversion an output file will be written)

found a ~2GB qwenimage.pt image, seems to be incomplete or something, can't convert it to Safetensor, getting error message incomplete, have to abort this experiment, and mark this conversion as failed from my side (in binary view, as I didn't got the expected model, on other hand partially failed, as conversion was working, but didn't got the expected result of a model to share). It created some evaluation images, they seems to be good, so I assume the Quantization worked as expected, but don't now how to get modell. Even the ~18 hours for time of conversion are to long in comparison with different Quantization methods. I.e Quantization of other modells (llm) ~10 min for Q4_0 / Q4_K

So got it converted, but no model to share, as pt file is useless, I assume after evaluating it will writes a proper safetensor file somewhere, but I don't want to wait for this 503 Days and 5 hours, to get this file.

My programming skills are basic. (I am an expert in “Hello World” with changing time of day greetings; anything beyond this knowledge is usually disastrous ;) ), unless you count simple debugging skills, which I can also do if I understand what the error message is saying.

Created Evaluation Images from Quantized Model:

EDIT (found evaluation Images before Quantization):

If you compare the reflection the first and the second noodle soup, you will notice small differences in reflection of the light, superb result (if I could use the model :P)

[Milor123]

today morning 01:32 a.m. local time, my quantization was finished, but after this step the script started with an evaluation of the model, time till it will be finnished -> approx 12077 hours

In my opinion totally inefficient. (it seems there is some where an option to switch this option off, but don't know where to set, and event he option is set, I don't know if after a conversion an output file will be written)

found a ~2GB qwenimage.pt image, seems to be incomplete or something, can't convert it to Safetensor, getting error message incomplete, have to abort this experiment, and mark this conversion as failed from my side (in binary view, as I didn't got the expected model, on other hand partially failed, as conversion was working, but didn't got the expected result of a model to share). It created some evaluation images, they seems to be good, so I assume the Quantization worked as expected, but don't now how to get modell. Even the ~18 hours for time of conversion are to long in comparison with different Quantization methods. I.e Quantization of other modells (llm) ~10 min for Q4_0 / Q4_K

So got it converted, but no model to share, as pt file is useless, I assume after evaluating it will writes a proper safetensor file somewhere, but I don't want to wait for this 503 Days and 5 hours, to get this file.

My programming skills are basic. (I am an expert in “Hello World” with changing time of day greetings; anything beyond this knowledge is usually disastrous ;) ), unless you count simple debugging skills, which I can also do if I understand what the error message is saying.

Created Evaluation Images from Quantized Model:

EDIT (found evaluation Images before Quantization):

If you compare the reflection the first and the second noodle soup, you will notice small differences in reflection of the light, superb result (if I could use the model :P)

Thank you for sharing, if you manage to find the correct way to do it, please share the configuration files and experience, thank you very much!

[emogie]

After reading documentation again, again and again (yep I have issues with reading and with understanding, that why I'm not very well in programming, besides of issue that I never understood the class system in non procedural programming languages ;(.

Will start tomorrow morning a next try with following command:
python -m deepcompressor.app.diffusion.ptq examples/diffusion/configs/model/qwen-image-2512.yaml examples/diffusion/configs/svdquant/int4.yaml examples/diffusion/configs/svdquant/fast.yaml --eval-benchmarks MJHQ --eval-num-samples 1 --save-model /gguf

the additional commands:
--eval-benchmarks MJHQ --eval-num-samples 1 <- I tried to skip this in previous quantization, without this command default 5000 images with this command 1 Image evaluation (2 hours additionally waiting is ok)
--save-model /gguf <- to save the model in folder gguf (I don't know, if it creates the folder automatically, so I will create it manually, in root directory -> like C:\deepcompressor-ussoewwin\gguf
examples/diffusion/configs/svdquant/fast.yaml <- this should speed up the things, due to documentation
additionally I made some changes in the default qwen-image.yaml and reduced the batch sizes from 8 to 2, this should speed up the things additionally, as VRAM usage was 100% and GPU Usage ~98-99% (but I need to tell at this point, that I made a threshold so max GPU usage is at 95%)

Test run says, with new config runtime ~6 hours on my 4060 RTX 16GB, with 5060 it should be the half of the time with DDR5 Ram.

config attached
qwen-image-2512.yaml

[emogie]

after quantization, evaluation and different further steps, round about 21,5 hours runtime, it aborted with:
ModuleNotFoundError: No module named 'ImageReward'

no quantized model created.

I wonder why poetry didn't installed this package as it is included in pyproject.toml

---

EDIT: Jan 10, 2026

---

now it failed with error message

** File "C:\deepcompressor-ussoewwin\deepcompressor\app\diffusion\ptq.py", line 665, in ptq
os.symlink(os.path.relpath(branch_load_from, save_dirpath), save_path.branch)
FileExistsError: [WinError 183] A file cannot be created if it already exists: '..\deepcompressor-ussoewwin\runs\diffusion\cache\quant\qdiff.2\branch\w.4-x.4-y.16\w.sint4-x.sint4.u-y.bf16\w.v64.bf16-x.v64.bf16-y.tnsr.bf16\w.skip.[e+rs+rtp+s+tm+tpi+tpo]-x.skip.[e+rs+rtp+s+tan+tm+tn+tpi+tpo]-y.skip.[]\shift\lowrank.r128.compensate\lowrank.OutputsError.Manual.Layer.d2.en1.sn1\lowrank.i1.r128.compensate\lowrank.skip.[rc+tan+tn]\qwenimage-edit-2512.pt' -> '/gguf\branch.pt'**

mentioned file do not exists in gguf folder.
took file qwenimage-edit-2512.pt from source the folder, again only a 2GB file

tried to convert the output file

command:
python -m deepcompressor.backend.nunchaku.convert --quant-path ./qwenimage-edit-2512.pt --output-root ./gguf --model-name qwenimage-edit-2512

error:
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "C:\deepcompressor-ussoewwin\deepcompressor\backend\nunchaku\convert.py", line 436, in
assert "flux" in model_name.lower(), "Only Flux models are supported."

I had a look in this file, it seems it only supports flux

---

also tried as name "flux", but than it requests the file scale.pt

--> IGNORE THIS: At this point giving up, sorry, I don't have any clue, what to do next.

uploaded at huggingface.co
image-edit-2512.pt
fast.yaml
image-edit-2512.yaml
int4.yaml

https://huggingface.co/emogie3D/experiment_qwen-image-2512-PT-File/

---

EDIT2: Jan 10, 2026

Found on my C Drive in an unexpected location the GGUF files
instead of C:\deepcompressor-ussoewwin\gguf the files were placed to C:\gguf
First to mentioned, there were created 4 Files in PT format, as I already created a repository for PT Files, I will upload them to same location -> https://huggingface.co/emogie3D/experiment_qwen-image-2512-PT-File/

branch.pt as symlink to location of the former mentioned 2GB File (in Windows currently symlinks are not supported - that's why it needs to be copied manually), this should be named to branch.pt

additional 3 further files
model.pt ~ 40GB
scale.pt - 656 MB
wgts.pt - 52KB

these created files are due to the timestamp are from my test which I started the day before yesterday and were finished yesterday, the conversion from yesterday till today got lost.

could take some time with this file size to upload them.

These needs to be converted to safetensor format somehow, I will try later with mentioned command:

python -m deepcompressor.backend.nunchaku.convert --quant-path ./qwenimage-edit-2512.pt --output-root ./gguf --model-name qwenimage-edit-2512

but need breakfast first ;) and my workstation needs a repairinstallation, after upload of the mentioned files and my breakfast.

[vgabbo]

@emogie congrats for your perseverance!! I'm following your updates :)