Convolutional-Neural-Networks

MULTISCALE LARGE KERNEL CNNS FOR IMAGE CLASSIFICATION

Authors: Umar Khayaz, Khalid Khwaji, River F Pease, Allen Gao

Notes

This repository is intended for:

• research experiments on multi-scale kernel CNNs
• comparisons against baseline CNNs, large-kernel and multi-scale kernel CNNs, and ViTs
• studying receptive field design in small-data image classification

Each folder holds one of the 4 self-contained models for image classification.

1. DeiT-Tiny: CIFAR-100 Fine-Tuning

• Model: DeiT-Tiny (deit_tiny_patch16_224) pretrained on ImageNet-1K, fine-tuned on CIFAR-100 via the timm library. Classification head replaced for 100-class output. Trained with AdamW, cosine LR annealing, and label smoothing over 17 epochs.
• Environment: Google Colab (T4 GPU) with PyTorch and timm.
• Reference: DeiT architecture from Touvron et al., 2020. Model weights loaded via PyTorch Image Models (timm). Fine-tuning pipeline written from scratch for this project.

2. RepLKNet Implementation

This repository contains a direct implementation of the paper:

Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs
Xiaohan Ding et al., CVPR 2022

The implementation is based on the RepLKNet design philosophy of using very large convolution kernels to improve receptive field size and capture global spatial information without relying on attention mechanisms. This version taken from Ding et al. is adapted for our image classification experiments and supports training on CIFAR-100 with configurable stage kernel sizes.

Features

• Direct RepLKNet-style implementation based on the reference paper
• Configurable large kernel combinations (for example 3,7,15)
• CIFAR-100 training and evaluation support
• Training/validation logging and curve plotting
• Compatible with mixed precision training

Running

This code can be trained easily on Google Colab using:

• T4 GPU
• A100 GPU

A100 is recommended for faster training, while T4 is sufficient for standard experimentation.

Example

python main.py \
  --data_set CIFAR \
  --data_path ./data \
  --model RepLKNet-31B \
  --input_size 32 \
  --test_size 32 \
  --nb_classes 100 \
  --stage_kernels 3,15 \
  --batch_size 64 \
  --epochs 200 \
  --opt adamw \
  --lr 5e-4 \
  --weight_decay 0.05 \
  --drop_path 0.3 \
  --use_amp True