Skip to content

Commit f905170

Browse files
committed
eager: warn once when int4_tensorwise runs the slow reference path on CUDA
A missing compiled extension silently drops int4_tensorwise to the eager reference (~4x slower than the int8 kernels), which reads as 'int4 is slow' rather than 'the kernels are not installed' (reported on a 4070 in PR #63: 4.27 s/it eager vs the expected ~1 s/it). Warn once with the actual reason: extension missing vs fused-path constraints unmet.
1 parent 3e94fd4 commit f905170

1 file changed

Lines changed: 27 additions & 0 deletions

File tree

comfy_kitchen/backends/eager/quantization.py

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1292,6 +1292,9 @@ def dequantize_int4_convrot_weight_dtype(
12921292
return dequantize_int4_convrot_weight(q, scale, group_size).to(DTYPE_CODE_TO_DTYPE[output_dtype_code])
12931293

12941294

1295+
_int4_eager_cuda_warned = False
1296+
1297+
12951298
def int4_linear(
12961299
x: torch.Tensor,
12971300
weight: torch.Tensor,
@@ -1322,6 +1325,30 @@ def int4_linear(
13221325
Returns:
13231326
Result tensor [..., N].
13241327
"""
1328+
global _int4_eager_cuda_warned
1329+
if x.is_cuda and not _int4_eager_cuda_warned:
1330+
_int4_eager_cuda_warned = True
1331+
import logging
1332+
try:
1333+
from comfy_kitchen.backends.cuda import _C as _cuda_ext
1334+
_has_kernels = hasattr(_cuda_ext, "int4_tensorwise_quantize")
1335+
except Exception:
1336+
_has_kernels = False
1337+
if _has_kernels:
1338+
reason = (
1339+
"the compiled kernels are present but this call does not meet the "
1340+
"fused path's constraints (SM >= 8.0, K % 64 == 0)"
1341+
)
1342+
else:
1343+
reason = (
1344+
"the compiled comfy_kitchen CUDA extension with the int4 kernels was "
1345+
"not found — build it from this checkout "
1346+
"(python setup.py build_ext --inplace) or install a wheel that ships it"
1347+
)
1348+
logging.warning(
1349+
"int4_tensorwise is running on the SLOW eager reference path on a CUDA "
1350+
"device (roughly 4x slower than the int8 kernels): %s.", reason
1351+
)
13251352
w_codes = _int4_unpack_rowwise(weight.to(device=x.device).contiguous())
13261353
if x.shape[-1] != w_codes.shape[-1]:
13271354
raise ValueError(

0 commit comments

Comments
 (0)