Zero-Shot Anomaly Detection using Vision Foundation Models

This project implements zero-shot and few-shot anomaly detection on the MVTec Anomaly Detection (AD) dataset using the pre-trained DINOv3 vision foundation model.

The code is tested with Python 3.12.12. Other Python versions may lead to dependency conflicts, so it is strongly recommended to install Python 3.12.12 before running the project.

---

Setup

1. Download Required Files

You need two external resources:

• MVTec AD Dataset
  Download: https://www.mvtec.com/company/research/datasets/mvtec-ad/downloads

• DINOv3 Weights (Meta AI)
  Download: https://ai.meta.com/resources/models-and-libraries/dinov3-downloads/

DINOv3 Weights

1. Follow the DINOv3 download link.

2. Fill in the Meta AI download form.

3. After approval, you will receive an e-mail with download links.

4. Download the file named:

   dinov3_vitb16_pretrain_lvd1689m-73cec8be.pth

5. Rename it to:

   dinov3_vitb16_distilled.pth

6. Place this file in the project root folder (same level as project_DL.ipynb).

The notebook expects this exact path:

WEIGHTS_PATH = "dinov3_vitb16_distilled.pth"

Dataset Structure

You can either download the full MVTec AD dataset or selected categories. The expected directory structure is:

root
│
├─ project_DL.ipynb
├─ dinov3_vitb16_distilled.pth
└─ mvtec_anomaly_detection
   ├─ bottle
   ├─ cable
   ├─ capsule
   ├─ carpet
   ├─ grid
   ├─ hazelnut
   ├─ leather
   ├─ metal_nut
   ├─ pill
   ├─ screw
   ├─ tile
   ├─ toothbrush
   ├─ transistor
   ├─ wood
   └─ zipper

If you only download some categories, make sure they still live under:

./mvtec_anomaly_detection/<category_name>/

2. Install Dependencies

All required libraries are listed in requirements.txt.

From the project root, run:

pip install -r requirements.txt

The code has been tested on Windows and macOS. A GPU (CUDA) is recommended for faster experiments, but the code will also run on CPU (slower, especially at 512×512).

Usage

All experiments are driven from the Jupyter notebook:

project_DL.ipynb

1.Image Size (Resolution)

You can control the input resolution via:

IMG_SIZE = 512  # or 256

• Must be a multiple of 16 (because ViT patch size = 16).
• 512×512
  • Better anomaly localization
  • Significantly slower and more memory-intensive
• 256×256
  • Faster
  • Slightly worse localization in many categories

2. Few-shot & Evaluation Parameters

The following parameters can be modified inside project_DL.ipynb to change the experiment setup:

k_values = [2, 4, 8]            # Number of anomaly images per defect type (few-shot levels)
n_repeats = 3                   # How many times to resample few-shot banks and average the results
metric = 'cosine'               # Similarity / distance metric ('cosine' or 'euclidean')
n_neighbors = 5                 # k-NN neighborhood size for aggregating patch distances/similarities
use_mask = True                 # If True, use ground-truth masks to keep only defective patches in the anomaly bank
target_categories = categories  # List of categories to include in the experiments

• Lower k_values → faster, less anomaly supervision
• Higher k_values → richer anomaly bank, but more risk of overfitting and slower runtime
• Increasing n_repeats → more stable averages, but longer total runtime
• Setting use_mask=False → uses all patches from anomalous images (simpler, but noisier)
• Lower n_neighbors → more sensitive to local defect details, but noisier predictions
• Higher n_neighbors → smoother anomaly scores and more stable predictions, but may blur small defects