[Bug]: distributed oneshot with kv_cache_scheme crashes in greedy_bin_packing on weightless attention modules

[Abhishek-chohan]

⚙️ Your current environment

• llm-compressor: main @ f0f20063d
• compressed-tensors: main (with is_cached_attention_module from Fix KV cache quantization for composite multimodal models compressed-tensors#780)
• torch 2.10.0, transformers 5.10.1
• Hardware: 2x CUDA GPUs, torchrun --nproc_per_node=2

🐛 Describe the bug

Running oneshot with the distributed pipeline (device_map="auto_offload" under torchrun) and a kv_cache_scheme crashes at the first sequential epoch boundary in QuantizationModifier.on_sequential_epoch_end:

File "src/llmcompressor/modifiers/quantization/quantization/base.py", line 100, in on_sequential_epoch_end
    module_list, rank_to_modules, module_to_rank = greedy_bin_packing(
File "src/compressed_tensors/distributed/assign.py", line 33, in greedy_bin_packing
    items.sort(key=item_weight_fn, reverse=True)
File "src/llmcompressor/modifiers/quantization/quantization/base.py", line 103, in <lambda>
    item_weight_fn=lambda mod: mod.weight.numel(),
AttributeError: 'Qwen3MoeAttention' object has no attribute 'weight'

kv_cache_scheme is converted to an input-activation-only scheme (weights=None) and attached directly to the attention containers matched by is_cached_attention_module (vllm-project/compressed-tensors#780). Those modules pass is_module_quantized (via input_activations is not None), so they land in the quantized-module list, but they have no .weight for the weight-calibration bin packing.

Filtering the containers out of the module list earlier is not a fix either: sync_obs_act_stats and update_qparams(..., ACTIVATION_OBS) run on that same list and are what finalize k_scale/v_scale on those modules. Dropping the containers before the sync leaves the checkpoint with uninitialized kv scales; we hit this in practice before settling on the approach below.

Proposed fix: keep the quantized-module list intact for the activation-stat sync, but restrict the bin packing to modules that carry a weight tensor. Single-GPU runs are unaffected (no packing on that path).

Related: #2949 (distributed offload), #2853 (zero kv scales via a different path).

🛠️ Steps to reproduce

torchrun --nproc_per_node=2 on any CUDA pair:

from compressed_tensors.quantization import QuantizationArgs
from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import QuantizationModifier

oneshot(
    model="nm-testing/tinysmokeqwen3moe",
    dataset=...,  # 16 samples, max_seq_length=512
    recipe=QuantizationModifier(
        scheme={"W4A16": ["Linear"]},
        ignore=["lm_head"],
        kv_cache_scheme=QuantizationArgs(
            num_bits=8, type="float", symmetric=True, strategy="tensor"
        ),
    ),
    max_seq_length=512,
    num_calibration_samples=16,
    pipeline="independent",
)

with the model loaded via AutoModelForCausalLM.from_pretrained(..., dtype=torch.bfloat16, device_map="auto_offload"). The single-GPU reference run completes and produces finite, positive k/v scales; the 2-rank run crashes as above.

I have a fix and a 2-rank regression test (test_ddp_smoke_rtn_kv_cache, asserts the DDP kv scales are finalized and match the single-GPU reference) ready locally, and will open a PR alongside this issue.