[Bug]: get_input_embeddings not auto‑handled for Qwen3OmniMoeForConditionalGeneration #1872
Description
⚙️ Your current environment
The output of python collect_env.py
### Environment Information ###
Operating System: `Linux-6.16.5-100.fc41.x86_64-x86_64-with-glibc2.40`
Python Version: `3.12.11 (main, Jun 12 2025, 00:00:00) [GCC 14.3.1 20250523 (Red Hat 14.3.1-1)]`
llm-compressor Version: `0.7.1`
compressed-tensors Version: `0.11.0`
transformers Version: `4.57.0.dev0`
torch Version: `2.8.0`
CUDA Devices: `['NVIDIA GeForce RTX 5070 Ti', 'NVIDIA GeForce RTX 5070 Ti', 'NVIDIA GeForce RTX 5070 Ti', 'NVIDIA GeForce RTX 5070 Ti']`
🐛 Describe the bug
Trying to quantize the new Qwen3-Omni model and getting the following error. Not sure if the root cause is in the transformers library code or in llm-compressor.
File ".../quantize.py", line 62, in quantize_model
oneshot(model=model, recipe=recipe) # Applies the quantization recipe
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
File ".../llmcompressor/entrypoints/utils.py", line 65, in pre_process
untie_word_embeddings(model_args.model)
File ".../llmcompressor/transformers/sparsification/compressed_tensors_utils.py", line 131, in untie_word_embeddings
input_embed = model.get_input_embeddings()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../transformers/modeling_utils.py", line 1802, in get_input_embeddings
NotImplementedError: `get_input_embeddings` not auto‑handled for Qwen3OmniMoeForConditionalGeneration; please override in the subclass.
Can anyone help out?
🛠️ Steps to reproduce
...
model = Qwen3OmniMoeForConditionalGeneration.from_pretrained(modelpath, torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained(modelpath)
ignore_list = 'lm_head'.split()
recipe = QuantizationModifier(targets="Linear", scheme='FP8_DYNAMIC', ignore=ignore_list)
oneshot(model=model, recipe=recipe)