Skip to content

README.md

Latest commit

 

History

History
46 lines (30 loc) · 1.45 KB

README.md

File metadata and controls

46 lines (30 loc) · 1.45 KB

Qwen 3 VL 4B Instruct FP8 Model Setup

Step-by-Step Instructions

1. Create Model Folder in Backend.AI

First, create a model folder in Backend.AI with the name that matches the --local-dir parameter (without the /home/work prefix).

Model Folder Name: qwen3-vl-4b-instruct-fp8

  • Go to your Backend.AI console
  • Navigate to the Storage/Folders section
  • Create a new folder named qwen3-vl-4b-instruct-fp8
  • Set the folder type to "Model"

2. Create a Batch Session

Create a batch-type session and mount the model folder:

  • Session Type: Batch
  • Mount the qwen3-vl-4b-instruct-fp8 folder to /home/work/qwen3-vl-4b-instruct-fp8

3. Download the Model

Once you have your batch session running with the mounted folder, execute the following command:

hf download --local-dir /home/work/qwen3-vl-4b-instruct-fp8 Qwen/Qwen3-VL-4B-Instruct-FP8

Important: The --local-dir path must match where you mounted your model folder.

Notes

  • The model will be downloaded from Hugging Face Hub
  • Ensure you have sufficient storage space for the 4B FP8 model
  • The download path /home/work/qwen3-vl-4b-instruct-fp8 should match your mounted folder path
  • This model is quantized in FP8 format for efficient inference

Model Details

  • Model Name: Qwen/Qwen3-VL-4B-Instruct-FP8
  • Type: Vision-Language Model (VLM)
  • Quantization: FP8
  • Parameters: 4 Billion
  • Use Case: Multimodal instruction following (text + image)