Roadmap

Currently under working

• COCO training using TF matching with Torch.
• Basic imagenet recognition training for PyTorch backend using train_script.py.
• Other CLIP attention output headers for text model, both PyTorch and Tensorflow.
• Basic text training for both TF and Torch. Extend to other format datasets like Github karpathy/llama2.c.
• Segment-all-model AutoMaskGenerator.
• Github facebookresearch/DiT or Github milmor/diffusion-transformer-keras.
• Github tensorflow/models MobileNetV4 or Github jaiwei98/MobileNetV4-pytorch.
• Github bytedance/next-vit.
• Github PaddlePaddle/pp_hgnet.
• Github microsoft/beit3.
• YOLOV9 / YOLOV10 / YOLOV11 / Github ultralytics/ultralytics or YOLOV11 Github jahongir7174/YOLOv11-pt.
• Other PyTorch Models.

2024.04.20 under working

• Segment-all-model basic implementation.
  • Github ChaoningZhang/MobileSAM. - 2023.11.28
  • Github CASIA-IVA-Lab/FastSAM.
  • Github mit-han-lab/efficientvit. - 2023.12.02
  • Github xinghaochen/TinySAM. - 2023.12.27
• Update from Github NVlabs/GCVit. - 2023.11.23
• Update from Github THU-MIG/RepViT. - 2023.11.23
• Port LLaMA2_1B from Github jzhang38/TinyLlama. - 2023.12.16
• Port CSPNeXt from Github open-mmlab/mmdetection/rtmdet. - 2024.01.13
• Add torch_coco_train_script.py matching with Github ultralytics/ultralytics, also supporting using self defined dataset and validator. - 2024.02.28
• Fit TF 2.16.- 2024.03.16
• Port YOLOV8 segmentation models. - 2024.04.20
• Github AILab-CVC/GroupMixFormer.
• Github wongkinyiu/yolov9.

2023.11.20 under working

• Distillation training.
• CLIP training using PyTorch and Tensorflow.
  • Basic CLIP training. - 2023.07.25
  • PyTorch training script. - 2023.07.28
  • Match Tensorflow training results with PyTroch one. - 2023.09.09
  • Training scipt reformate. - 2023.09.09
• Basic text training for both TF and Torch.
  • Basic text training for both Torch. - 2023.09.09
  • Basic text training for both TF. - 2023.09.09
• Port MetaTransformer from Github invictus717/MetaTransformer. - 2023.07.28
• Port EfficientNetEdgeTPU from Github tensorflow/tpu/edgetpu. - 2023.07.29
• Port RepViT from Github THU-MIG/RepViT. - 2023.08.02
• Port LLaMA2 tiny-story weights from Github karpathy/llama2.c. - 2023.08.05
• Port FastViT from Github apple/ml-fastvit. - 2023.08.22
• Port efficientvit_b L models. - 2023.09.24
• Port Stable Diffusion v1.5 model. - 2023.09.27
• Update for DDPM training results for both TF and Torch - 2023.11.16
• Github open-mmlab/mmpretrain/riformer.
• grid_sample and deformable_conv implementation.
• Github FishAndWasabi/YOLO-MS.

2023.07.20 under working

• YOLOV8 COCO training using PyTorch.
  • Port COCO and Imagenet models for prediction. - 2023.04.08
  • Model training using PyTorch and ultralytics dataset and validator. - 2023.04.27
• Port Github JierunChen/FasterNet. - 2023.03.23
• Port Github sail-sg/inceptionnext. - 2023.04.05
• Port convnext_xxlarge from timm. - 2023.04.15
• Port DINOv2 from Github facebookresearch/dinov2. - 2023.04.26
• Port Eva02 from timm and Github baaivision/EVA. - 2023.04.28
• Port DiNAT from Github SHI-Labs/Neighborhood-Attention-Transformer/DiNAT. - 2023.05.10
• Port YOLO-NAS from Github Deci-AI/super-gradients. - 2023.05.13
• Port EfficientViT_M from Github microsoft/Cream/EfficientViT. - 2023.05.24
• Port VanillaNet from Github huawei-noah/VanillaNet. - 2023.05.26
• Port EfficientViT_B from Github mit-han-lab/efficientvit. - 2023.05.30
• Port GPT2 for further usage. - 2023.06.03
• Update T4 inference qps for most models. - 2023.06.03
• Port Hiera from Github facebookresearch/hiera. - 2023.06.11
• Port FasterViT from Github NVlabs/FasterViT. - 2023.06.19

2023.01.20 under working

• Port Keras MaxViT from Github google-research/maxvit. - 2022.10.20
• CoAtNet0 224 training 300 epoches 82.21 -> 82.79. - 2022.11.12
• Port Keras GhostNetV2 from Gitee mindspore/models/ghostnetv2. - 2022.11.18
• Port Keras YOLOV7 from Github WongKinYiu/yolov7. - 2022.12.01
• Port Keras ConvNeXtV2 from Github facebookresearch/ConvNeXt-V2. - 2023.01.08
• Port Keras EfficientFormerV2 from Github snap-research/efficientformer. - 2023.01.09
• Port Keras BEiTV2 from Github microsoft/beit2. - 2023.01.10
• Port Keras EVA from Github baaivision/EVA. - 2023.01.11
• Port Keras FlexiViT from Github google-research/flexivit. - 2023.01.11
• Support FlexiViT changing patch_size. - 2023.01.12
• Port Keras PVT_V2 from Github whai362/PVT. - 2023.01.12
• Add PVT_V2B2_linear. - 2023.01.15
• Port Keras IFormer from Github sail-sg/iFormer. - 2023.01.15
• Port GhostNet_050 / GhostNet_130 and LCNet_050 / LCNet_100 / LCNet_130 ssld weights from Github PaddlePaddle/PaddleClas. - 2023.01.15
• Port Keras GPViT from Github ChenhongyiYang/GPViT. - 2023.01.15
• Port Keras CAFormer and ConvFormer from Github sail-sg/metaformer. - 2023.01.18
• Port Keras TinyViT from Github microsoft/TinyViT. - 2023.01.18
• Port Keras MogaNet from Github Westlake-AI/MogaNet. - 2023.01.18

2022.09.20 under working

• Token label basic training test. Including related random_flip_left_right / cutmix / mixup / random_crop_and_resize for token_label data. This is a large change in data.py. - 2022.04.30
• Token label further improvement.
• Distillation basic implementation. - 2022.05.05
• CMTTiny fine-tune 224. - 2022.05.07
• SwinTransformerV2Tiny_ns training 160 -> 224. - 2022.05.18
• Retrain CoAtNet0 using vv_dim = key_dim. 160 accuracy 80.50 -> 80.48, 224 accuracy 82.23 -> 82.21, just acceptable. - 2022.04.24
• Port ConvNeXtTiny / ConvNeXtSmall + 224 / 384 + imagenet21k weights from Github facebookresearch/ConvNeXt. - 2022.04.22
• Port Keras DaViT from Github dingmyu/davit. - 2022.04.26
• Port Keras NAT from Github SHI-Labs/Neighborhood-Attention-Transformer. - 2022.05.11
• Port Keras SwinTransformerV2 new weights from Github microsoft/Swin-Transformer. - 2022.05.16
• Port CMTTiny_torch / CMTSmall_torch / CMTBig_torch from Github ggjy/CMT.pytorch. - 2022.06.30
• Port Keras EdgeNeXt from Github mmaaz60/EdgeNeXt. - 2022.07.15
• Port Keras GCVit from Github NVlabs/GCVit. - 2022.07.19
• Port Keras EfficientFormer from Github snap-research/efficientformer. - 2022.07.23
• Port Keras MobileViT_V2 from Github apple/ml-cvnets. - 2022.07.25
• Port Keras HorNet from Github raoyongming/hornet. - 2022.09.12

2022.04.20 under working

• Modified using anchors_mode in [anchor_free, yolor, efficientdet] instead of all previous use_anchor_free_mode and use_yolor_anchors_mode in training / evaluating / model structure. - 2022.04.15
• YOLOR training strategy.
  • Perspective and scale random_perspective. - 2022.03.25
  • Anchor assign build_targets. - 2022.03.25
  • BCE + Focal loss compute_loss. - 2022.03.25
  • Basic train test. - 2022.04.19
• Training custom dataset. - 2022.03.29
• Port Keras WaveMLP from Github huawei-noah/wavemlp_pytorch. - 2022.03.30
• Port Keras MobileViT from Github apple/ml-cvnets/mobilevit. - 2022.04.02
• Port from timm: regnety_040 / regnety_064 / regnety_080 / regnetz_c16_evos / regnetz_d8_evos. - 2022.04.05
• Port Keras SwinTransformerV2 from timm swin_transformer_v2_cr. - 2022.04.06
• Fix YOLOX COCO evaluation. - 2022.03.28
• Fix YOLOR COCO evaluation. - 2022.04.11
• Port from timm: mobilenetv3 / fbnetv3 / lcnet / tinynet. LCNet150 / LCNet200 / LCNet250 from Github PaddlePaddle/PaddleClas - 2022.04.13
• Port from timm: ResNeXt101W_64 / EfficientNetV2T_GC. - 2022.04.20
• aotnet50 + evonorm + A3 training test. Accuracy improved from basic 78.47 -> 78.66 on 224. - 2022.04.18

2022.03.20 under working

• YOLOX training strategy.
  • Share / rotate / translate bboxes. - 2022.02.23
  • Mosaic augment. - 2022.02.24
  • Anchor free mode, dynamically assign foreground / background. - 2022.03.02
  • l1_target_loss / loss_iou / loss_obj / loss_cls. - 2022.03.07
  • Basic train test. - 2022.03.13
• Fine-tune CoAtNet A3 160 to 224 using magnitude 15. Accuracy improved from 82.06 -> 82.23. - 2022.03.08
• Port Keras UniFormer from Github Sense-X/UniFormer. - 2022.03.11
• Keras CMT training, 305 epochs. - 2022.03.21
• YOLOR model architecture and converted weights, CSP / CSPX. - 2022.03.16
• YOLOR anchors and prediction. - 2022.03.18
• Port YOLOR weights P6 / W6 / E6 / D6. - 2022.03.19
• Port efficientdet weights from automl efficientdet/Det-AdvProp.
• Port efficientnetv2_ds weights from rwightman/efficientdet-pytorch.

2022.02.20 under working.

• Update BEIT MultiHeadRelativePositionalEmbedding w/o transpose. Be caution of this update if wanna reload earlier own pre-trained BEIT / CoAtNet models.
• Re-format model weights names with multiple input resolutions.
• COCO AP / AR evaluation.
• Upload EfficientDetLite models.
• CoAtNet A3 160 + weight_decay excluding positional_embedding. Accuracy improved from 80.19 -> 80.50.
• Fine-tune CoAtNet A3 160 to 224 using magnitude 10 + weight_decay excluding positional_embedding. Accuracy improved from 81.99 -> 82.06.
• Upload YOLOX model architecture and converted weights.
• YOLOX prediction.

2022.01.20 under working

• Upload EfficientDet.
• COCO dataset load and FocalLossWithBbox.
• set anti_alias as default True for train and test. imagenet#comparing-resize-methods-bicubic-or-bilinear.
• Keras CoAtNet training, CoAtNet0 224.
• Imagenet training readme document.
• Trying COCO training...

2021.12.20 under working

• Port recent timm halonets and botnets.
  • eca_halonext26ts
  • sebotnet33ts_256
  • halonet50ts
  • halo2botnet50ts_256
  • regnetz_d8
  • regnetz_e8
  • vit_base_patch8_224
• Clean code for imagenet training.
• Merge Github leondgarse/keras_efficientnet_v2.
• Reproduce ResNet Strikes Back A2 result.
• Upload Keras CoAtNet0 160 A3 result.
• Better support for TFLite. Add model_surgery functions for TFLite conversion, including gelu / extract_pathes / group Conv2D.
• Keras CMT training.
• Some compare on Keras CoAtNet architecture.
• CoATNet0 A3 with drop_connect_rate=0.2 and 0.05.
• reload_model_weights_with_mismatch for HaloNet.
• Progressive training test on cifar10.
• Progressive training test on imagenet, input [96, 128, 160], A3.
• Learning rate cosine decay with restart training test on imagenet, --lr_decay_steps 33.

2021.10.20 under working

• Keras HaloNet HaloNet50 weights reload from timm releases/tag/v0.1-attn-weights.
• Port beit from microsoft/beit.
• Port RegNetZ from timm.
• Update HaloNet / BotNet / CotNet / ResNest using AotNet, like timm using byobnet / byoanet.
• Port BotNextECA26T / HaloNextECA26T / RegNetZ / HaloRegNetZB from timm.
• Update imagenet training data augment Randaug / RRC / cutmix / RandomErasing and others according to timm.
• Reproduce ResNet Strikes Back A3 result.
• Port model from ResNet Strikes Back released in timm releases/tag/v0.1-rsb-weights and others.
• Port RegNet from timm/models/regnet.py.

Other PyTorch models

• Github microsoft/CvT.
• Github microsoft/CSWin-Transformer.
• Github ShoufaChen/CycleMLP.
• Github pengzhiliang/Conformer.
• Github LeapLabTHU/DAT.
• Github facebookresearch/cait_models.
• Github UKPLab/sentence-transformers.
• Github Annbless/ViTAE.
• Github ylingfeng/DynamicMLP.
• Github microsoft/DynamicHead.
• Github IDEACVR/DINO.
• Github hustvl/TopFormer.
• Github dqshuai/metaformer.
• Github PaddleClas/mixformer.
• Github idstcv/ZenNAS.
• Github huawei-noah/Efficient-AI-Backbones/vig_pytorch.
• Github yolov6/models.
• Github OpenGVLab/InternImage.
• Github HVision-NKU/Conv2Former.
• Github zhangzjn/EMO.
• Github stevenlauhkhk/large-separable-kernel-attention
• Github qhfan/RMT.
• Github badripatro/svt.
• Github tany0699/FMViT.
• Github DaiShiResearch/TransNeXt.
• 2307.06304 Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution.
• 2401.03540 SeTformer is What You Need for Vision and Language