Optimize functionality using torch.compile

[dsikka]

Functionality in llm-compressor and compressed-tensors could be further optimized using triton jits and torch.compile to provide execution speed-ups. Some functionality that could potentially be replaced include:

1. Quantization Compressors compress_weight and decompress_weight functionality

• packed-compressor
• nvfp-compressor

2. Observer calculate_qparams functionality for the MinMax observer and MSE observer
3. Update GPTQ

A proposed solution should swap the existing code with optimized functionality, include updated tests, and quick benchmarks showing the differences in performance

[dsikka]

Hey for sure! Thank you!
Let us know the functionality you will be targeting.

Thanks,
Dipika

[GOavi101]

Hey @dsikka , can i look into this

I am thinking below functionality:

1. GPTQ algorithm: Optimized accumulate_hessian with torch.compile
2. Observers: Optimized MinMax observer _get_min_max and MSE observer _grid_search_mse

[Bias92]

Hi, I'd like to work on torch.compile optimization for the following targets:

compress_weight / decompress_weight in packed-compressor and nvfp-compressor
Observer calculate_qparams (MinMax, MSE)

Will include runtime benchmarks. Happy to coordinate if anyone else is working on these.

[HDCharles]

someone is already working on the GPTQ side but not the observer one