igie

DeiT-tiny (IGIE)

Model Description

DeiT Tiny is a lightweight vision transformer designed for data-efficient learning. It achieves rapid training and high accuracy on small datasets through innovative attention distillation methods, while maintaining the simplicity and efficiency of the model.

Supported Environments

GPU	IXUCA SDK	Release	Branch
MR-V100	4.4.0	26.03	release/26.03
MR-V100	4.3.0	25.12	release/25.12

Note: 请切换到与您的 SDK 版本对应的 Release 分支进行测试。请勿直接在 master 分支上运行测试，因为 master 分支可能包含与您的本地 SDK 版本不兼容的最新更改。

切换分支命令示例：git checkout release/26.03

Model Preparation

Prepare Resources

Pretrained model: https://download.openmmlab.com/mmclassification/v0/deit/deit-tiny_pt-4xb256_in1k_20220218-13b382a0.pth

Dataset: https://www.image-net.org/download.php to download the validation dataset.

Install Dependencies

# Install libGL
## CentOS
yum install -y mesa-libGL
## Ubuntu
apt install -y libgl1-mesa-glx

pip3 install -r ../../igie_common/requirements.txt
pip3 install --no-build-isolation mmcv==1.5.3 mmcls==0.24.0

Model Conversion

# git clone mmpretrain
git clone -b v0.24.0 https://github.com/open-mmlab/mmpretrain.git

# export onnx model
python3 ../../igie_common/export_mmcls.py --cfg mmpretrain/configs/deit/deit-tiny_pt-4xb256_in1k.py --weight deit-tiny_pt-4xb256_in1k_20220218-13b382a0.pth --output deit_tiny.onnx

# Use onnxsim optimize onnx model
onnxsim deit_tiny.onnx deit_tiny_opt.onnx

Model Inference

export DATASETS_DIR=/Path/to/imagenet_val/
export RUN_DIR=../../igie_common/

FP16

# Accuracy
bash scripts/infer_deit_tiny_fp16_accuracy.sh
# Performance
bash scripts/infer_deit_tin_fp16_performance.sh

Model Results

Model	BatchSize	Precision	FPS	Top-1(%)	Top-5(%)
DeiT-tiny	32	FP16	2172.771	74.334	92.175

References

• mmpretrain