Skip to content

platform_compatibility.md

Latest commit

 

History

History
117 lines (76 loc) · 3.73 KB

platform_compatibility.md

File metadata and controls

117 lines (76 loc) · 3.73 KB
Raw

Platform Compatibility Guide

This guide provides detailed information about Model Quantizer's compatibility with different platforms and Python versions.

Tested Environments

As of version 0.3.1, Model Quantizer has been tested and confirmed working on the following environments:

Platform Python Version Status Notes
macOS Sonoma 14.2 3.11 ✅ Working Tested with GPTQ quantization
macOS Sonoma 14.2 3.12 ✅ Working Tested with GPTQ quantization
Windows 10/11 3.11 ✅ Working Tested with all quantization methods
Windows 10/11 3.12 ✅ Working Tested with all quantization methods
Linux (Ubuntu) 3.11 ✅ Working Tested with all quantization methods
Linux (Ubuntu) 3.12 ✅ Working Tested with all quantization methods

macOS-Specific Information

CPU-Only Operation

On macOS, Model Quantizer operates in CPU-only mode. When using the tool on macOS, always specify --device cpu in your commands:

python -m quantizer.cli microsoft/Phi-4-mini-instruct --bits 4 --method gptq --output-dir qmodels/phi4-mini-4bit --device cpu

Installation on macOS

For the most reliable installation on macOS, use the provided installation script:

chmod +x install_dependencies.sh
./install_dependencies.sh

This script ensures that all dependencies are installed in the correct order, with proper versions for macOS compatibility.

GPTQ Quantization on macOS

GPTQ quantization works on macOS with both Python 3.11 and 3.12. The key requirements are:

  • Installing optimum[gptq] (not just optimum)
  • Installing gptqmodel<2.1.0
  • Using the --device cpu flag

Windows-Specific Information

Installation on Windows

For the most reliable installation on Windows, use the provided batch script:

install_dependencies.bat

GPU Support on Windows

If you have an NVIDIA GPU, you can use it for quantization by specifying --device cuda:

python -m quantizer.cli microsoft/Phi-4-mini-instruct --bits 4 --method gptq --output-dir qmodels/phi4-mini-4bit --device cuda

Linux-Specific Information

Installation on Linux

For the most reliable installation on Linux, use the provided shell script:

chmod +x install_dependencies.sh
./install_dependencies.sh

GPU Support on Linux

If you have an NVIDIA GPU, you can use it for quantization by specifying --device cuda:

python -m quantizer.cli microsoft/Phi-4-mini-instruct --bits 4 --method gptq --output-dir qmodels/phi4-mini-4bit --device cuda

Troubleshooting Platform-Specific Issues

macOS Issues

If you encounter issues on macOS:

  1. Ensure you're using the --device cpu flag
  2. Verify that optimum[gptq] is installed, not just optimum
  3. Try reinstalling the dependencies using the installation script

Windows Issues

If you encounter issues on Windows:

  1. Ensure you have the correct CUDA version installed if using GPU
  2. Try running the command prompt as administrator when installing dependencies
  3. Verify that all dependencies are installed correctly

Linux Issues

If you encounter issues on Linux:

  1. Ensure you have the correct CUDA version installed if using GPU
  2. Check that you have the necessary system libraries installed
  3. Verify that all dependencies are installed correctly

Reporting Platform-Specific Issues

If you encounter platform-specific issues not covered in this guide, please report them on the GitHub issues page with the following information:

  1. Your operating system and version
  2. Your Python version
  3. The command you're trying to run
  4. The complete error message
  5. The output of pip list to show installed packages