🦖 DINOv2 x Geosciences 🌍

This is the official code repository for our study:

DINOv2 Rocks Geological Image Analysis: Classification, Segmentation, and Interpretability

• Authors: Florent Brondolo and Samuel Beaussant
• Paper: https://arxiv.org/abs/2407.18100

---

This study investigates the interpretability, classification, and segmentation of CT-scan images of rock samples, with a particular focus on the application of DINOv2 within Geosciences. We compared various segmentation techniques to evaluate their efficacy, efficiency, and adaptability in geological image analysis. The methods assessed include the Otsu thresholding method, clustering techniques (K-means and fuzzy C-means), a supervised machine learning approach (Random Forest), and deep learning methods (UNet and DINOv2). We tested these methods using ten binary sandstone datasets and three multi-class calcite datasets.

👾 Code

We provide the code in the form of standalone notebooks to facilitate the reproducibility of our results and make them accessible to all (even to GPU-poor people!). The names of all the notebooks are self-explanatory and reproduce a subset of the paper's results.

• The multi-Otsu algorithm has been adapted from the scikit-image library.
• The K-means algorithm has been adapted from the OpenCV library.
• The FCM has been adapted from this work.
• The UNet has been adapted from Liang et al. (2022).
• The resNet152 model is available here.

Data and preprocessing

The raw data used for our experiments are public and freely available:

• Sandstones: https://www.digitalrocksportal.org/projects/317.
• Carbonates: https://www.digitalrocksportal.org/projects/151.

Some notebooks expect data as a NumPy (npy) archive, while others require TIFF (tif) files. In any case, before running anything, download the data and store it in a Google Drive folder. Following this you can run the data_preprocessing.ipy notebook to transform the raw data into the required formats.

Some results

Here, we present a portion of our experimental results, highlighting the performance of seven models: ResNet152, four variations of DINOv2, and two iterations of a UNet. The four versions of DINOv2 include a frozen DINOv2 paired with either a linear or a complex convolutional head, and a LoRA fine-tuned DINOv2 also paired with the same heads. For the UNet models, we utilized the same backbone with two different feature sizes: small (n=32) and large (n=64).

The results clearly demonstrate the superior capability of DINOv2 in interpreting raw rock CT scans. Experimental parameters were consistent across all experiments: 1000 images for training (split between two rock datasets) and 500 images for validation (from a third rock sample). Hyperparameters were set identically across all training sessions.

Model weights

You have the option to either train the models from scratch or perform inference using our pre-trained checkpoints, which can be downloaded from this link. These weights are the product of training with the DINOv2-base backbone (768 features), fine-tuned using LoRA plus a convolutional head. The numbers in the folder name indicate how many images have been used for the training set. The model definition code is available here. Additionally, we provide weights for the DINOv2-base model fine-tuned with LoRA and coupled to a linear head. These weights have been used for PCA evaluation.

Found a bug?

If you spot a bug or have a problem running the code, please open an issue. If you have any questions or need further assistance, don't hesitate to contact Florent Brondolo ([email protected]) or Samuel Beaussant ([email protected]).

📚 Citation / Bibtex

If you use our code or find our work helpful, please consider citing it as follows:

@article{brondolo2024dinov2,
  title={DINOv2 Rocks Geological Image Analysis: Classification, Segmentation, and Interpretability},
  author={Brondolo, Florent and Beaussant, Samuel},
  journal={arXiv preprint arXiv:2407.18100},
  year={2024}
}