Pathway1: Lightly library (for benchmark comparision)
Lightly is a python package for self-supervised learning, making training more efficient, and provides benchmarks, which makes comparisions on public datasets more feasible.
This unified comparison paved the way for our batch experiments. Thus we present a method for code implementation (as shown in following table). Our code draws inspiration from the simplicity and continuity of Lightly, and it has been tailored specifically for multi-view scenarios in these experiments. This allows anyone to easily replicate our results and propose enhancements.
| Dataset | TensorBoard Link |
|---|---|
| CIFAR100 | Jupyter Notebook |
| CIFAR10 | Jupyter Notebook |
| STL-10 | Jupyter Notebook |
| TINYIMAGENET | Jupyter Notebook |
| ImageNette | Jupyter Notebook |
Pathway2: AutoSSL (for efficiency,ablation experiment section)
We provide an in-depth analysis and diagnostic tool via an automated open-sourced pipeline that we have dubbed AutoSSL. Below, we outline its core functionalities and its utilization in our research. The Documentation can be accessed using using the following gitbook documentation.
-
Configurable Modeling AutoSSL is designed with modularity in mind, allowing for the decomposition of models into a series of modular functions. This offers a high degree of flexibility in the experimental setup and requires only a single configuration file for specification. For instance, the primary differences between SimCLR and SimSiam—such as the stopping gradient and dimensions of specific layers can all be specified via this configuration file.
-
Metrics and Experimental Records Upon the completion of training, AutoSSL automatically records a comprehensive set of metrics useful for assessing the quality of self-supervised learning models. These include:
- Standard Deviation (STD) of views
- STD of batches
- STD of feature representations
- K-Nearest Neighbors (KNN) accuracy
- Linear classification accuracy
- Running time
- Forward propagation time
- Backward propagation time
- GPU utilization
By following the aforementioned methodologies, the experiments presented in this paper can be reliably replicated and further extended for in-depth analyses.
We uploaded all raw data involved in the paper.
- Intermediate/Metrics, Meta, Records_of_Experiment
- Intermediate/CheckPoint_of_Result_in_Tensorboard
- Intermediate/YAML_Config_of_Model
- Intermediate/Calculation_for_Benchmark_Comparision.csv
- Intermediate/Calculation_for_Transfer_Learning.xlsx
The original data without processing, including all detailed data training meta information, log, metrics, checkpoints, configuration etc,. All results are extracted and processed from them.
We provide collated TensorBoard records for various datasets used in our experiments. Below is a table that enumerates these datasets along with their corresponding TensorBoard links.
| Dataset | TensorBoard Link |
|---|---|
| CIFAR100 | Tensorboard |
| CIFAR10 | Tensorboard |
| STL-10 | Tensorboard |
| TINYIMAGENET | Tensorboard |
| ImageNette | Tensorboard |
These TensorBoard records can be useful for gaining insights into the intermediate stages of the experiments, aiding in the fine-tuning and analysis of the model's performance.
Comprehensive record of the accuracies are obtained from various methods applied across the datasets. This pinpoints the instances, when 90%, and 95% of the baseline accuray-convergence is reached, respectively. Additionally, there is a detailed assessment of each method's performance specifically when trained on the ImageNette dataset.
Please Check Directory Intermediate for more details
collated data in form of csv. Please Check Transfer_Learning.csv for more details