Skip to content

Latest commit

 

History

History
53 lines (44 loc) · 15.3 KB

File metadata and controls

53 lines (44 loc) · 15.3 KB

Benchmarks

We are working on creating pretrained weights with NetsPresso Trainer and our own resources. We base training recipes on the official repositories or original papers to replicate the performance of models.

For models that we have not yet trained with NetsPresso Trainer, we provide their pretrained weights from other awesome repositories. We have converted several models' weights into our own model architectures. Therefore, we denote models that provide weights obtained from other repositories by appending an "*" to the model's name in the benchmark performance table. If you are interested in the original models, refer to the remarks in the table. We appreciate all the original authors and we also do our best to make other values.

If you have a better recipe, please share with us anytime. We appreciate all efforts to improve our models

Classification

Dataset Model Weights Input shape Acc_1 Acc_5 Params FLOPs NetsPresso Remarks
ImageNet1K EfficientFormer-l1* download (224, 224) 80.03 94.90 11.84M 2.60G Supported snap-research/EfficientFormer
ImageNet1K MixNet-s download (224, 224) 75.08 92.32 4.13M 0.51G Supported -
ImageNet1K MixNet-m download (224, 224) 76.37 93.07 5.01M 0.71G Supported -
ImageNet1K MixNet-l download (224, 224) 80.78 95.23 7.33M 1.16G Supported -
ImageNet1K MobileNetV3-small* download (224, 224) 67.67 87.41 2.54M 0.12G Supported torchvision
ImageNet1K MobileNetV3-large* download (224, 224) 75.31 92.64 5.48M 0.45G Supported torchvision
ImageNet1K MobileNetV4-conv-small* download (224, 224) 73.74 91.39 3.77M 0.38G Supported timm
ImageNet1K MobileNetV4-conv-medium* download (224, 224) 79.09 94.67 9.72M 1.68G Supported timm
ImageNet1K MobileNetV4-conv-large* download (256, 256) 81.84 95.74 32.59M 5.72G Supported timm
ImageNet1K MobileNetV4-hybrid-medium* download (224, 224) 80.49 95.40 11.07M 1.96G Supported timm
ImageNet1K MobileNetV4-hybrid-large* download (384, 384) 83.80 96.72 37.76M 15.54G Supported timm
ImageNet1K MobileViT-s* download (256, 256) 78.21 94.13 5.58M 4.07G Supported No input z-norm, apple/ml-cvnets
ImageNet1K ResNet18 download (224, 224) 69.11 88.87 11.69M 3.64G Supported -
ImageNet1K ResNet34 download (224, 224) 72.42 90.87 21.80M 7.34G Supported -
ImageNet1K ResNet50 download (224, 224) 79.67 94.82 25.56M 8.22G Supported -
ImageNet1K ViT-tiny* download (224, 224) 72.90 91.17 5.70M 2.52G Supported No input z-norm, apple/ml-cvnets

Semantic segmentation

Dataset Model Weights Input shape mIoU Pixel acc Params FLOPs NetsPresso Remarks
ADE20K SegFormer-b0* download (512, ?) or (?, 512) 37.15 76.78 3.75M 17.01G Supported mmsegmentation, Resize short edge to 512
Cityscapes PIDNet-s* download (1024, 2048) 78.76 96.12 7.72M 95.03G Supported XuJiacong/PIDNet

Object detection

Dataset Model Weights Input shape mAP50 mAP75 mAP50:95 Params FLOPs NetsPresso Remarks
COCO-val RT-DETR_res18* download (640, 640) 65.77 52.75 48.49 20.18M 40.36G Supported No input z-norm, lyuwenyu/RT-DETR
COCO-val RT-DETR_res50* download (640, 640) 72.64 59.50 54.73 42.94M 138.36G Supported No input z-norm, lyuwenyu/RT-DETR
COCO-val yolov9-tiny download (640, 640) 50.03 38.63 36.02 2.44M 9.99G Supported No input z-norm
COCO-val yolov9-s* download (640, 640) 62.63 51.13 47.13 7.23M 26.87G Supported No input z-norm, YOLO
COCO-val yolov9-m* download (640, 640) 67.43 56.13 51.72 20.12M 77.08G Supported No input z-norm, YOLO
COCO-val yolov9-c* download (640, 640) 69.16 57.90 53.28 25.50M 103.17G Supported No input z-norm, YOLO
COCO-val YOLOX-nano* download (416, 416) 41.30 27.90 26.33 0.91M 1.08G Supported Megvii-BaseDetection/YOLOX, conf_thresh=0.01, nms_thresh=0.65
COCO-val YOLOX-tiny* download (416, 416) 50.69 36.18 34.00 5.06M 6.45G Supported Megvii-BaseDetection/YOLOX, conf_thresh=0.01, nms_thresh=0.65
COCO-val YOLOX-s download (640, 640) 58.56 44.10 40.63 8.97M 26.81G Supported conf_thresh=0.01, nms_thresh=0.65
COCO-val YOLOX-m* download (640, 640) 65.00 51.34 47.04 25.33M 73.76G Supported Megvii-BaseDetection/YOLOX, conf_thresh=0.01, nms_thresh=0.65
COCO-val YOLOX-l* download (640, 640) 68.07 55.18 50.68 54.21M 155.65G Supported Megvii-BaseDetection/YOLOX, conf_thresh=0.01, nms_thresh=0.65
COCO-val YOLOX-x* download (640, 640) 69.13 56.46 51.79 99.07M 281.94G Supported Megvii-BaseDetection/YOLOX, conf_thresh=0.01, nms_thresh=0.65
COCO-val YOLO-Fastest-v2 download (640, 640) 25.03 11.60 12.78 0.25M 0.74G Supported conf_thresh=0.01, nms_thresh=0.65