Need Guideance for NMT models Quantization and prunning

[syedhamza671]

I couldn't find any blog , tutorial or documentation for NeMo NMT model's quantization and prunning. If anyone can help it would mean a lot.

[ashors1]

Hi, thanks for the question. We provide general quantization and pruning support for LLMs (some resources: https://docs.nvidia.com/nemo-framework/user-guide/latest/model-optimization/quantization/quantization.html, https://docs.nvidia.com/nemo-framework/user-guide/latest/model-optimization/pruning/pruning.html, https://github.com/NVIDIA/NeMo/tree/main/tutorials/llm/llama/pruning-distillation). However, the focus of this work has been on encoder-only or decoder-only models.