AutoSSVH: Exploring Automated Frame Sampling for Efficient Self-Supervised Video Hashing

[toc]

1. Introduction

This repository contains the PyTorch implementation of our work at CVPR 2025:

AutoSSVH: Exploring Automated Frame Sampling for Efficient Self-Supervised Video Hashing. Niu Lian, Jun Li, Jinpeng Wang, Ruisheng Luo, Yaowei Wang, Shu-Tao Xia, Bin Chen.

We present "AutoSSVH", a novel approach in the field of self-supervised video hashing. "AutoSSVH" integrates an adversarial strategy-based sampler with Hash Voting, which accelerates adversarial convergence. Inspired by GANs, our sampler automatically selects frames that are difficult to reconstruct, thereby enhancing the semantic information of the hash codes. Hash Voting, based on ensemble learning, employs a P2Set paradigm to strengthen the neighborhood information of the hash codes. Extensive experiments demonstrate the efficiency and robustness of "AutoSSVH". We hope that our research provides valuable insights for future strategies in adversarial sampling.

Besides, we invite readers to refer to our previous work ConMH and S5VH.

In the following, we will guide you how to use this repository step by step. 🤗🐶

2. Preparation

git clone https://github.com/EliSpectre/CVPR25-AutoSSVH.git
cd CVPR25-AutoSSVH/

2.1 Requirements

• python==3.11.8
• numpy==1.26.4
• pytorch==2.0.1
• torchvision==0.15.2
• scipy==1.5.4
• h5py==3.1.0
• addict==2.4.0
• faiss==1.8.0

2.2 Download the video feature datasets and organize them properly

Before running the code, make sure that everything is ready. The working directory is expected to be organized as below:

CVPR25-AutoSSVH/

checkpoint/

• activitynet/
• AutoSSVH_16bit
• AutoSSVH_32bit
• AutoSSVH_64bit
fcv/
• AutoSSVH_16bit
• AutoSSVH_32bit
• AutoSSVH_64bit
hmdb/
• AutoSSVH_16bit
• AutoSSVH_32bit
• AutoSSVH_64bit
ucf/
• AutoSSVH_16bit
• AutoSSVH_32bit
• AutoSSVH_64bit

data/

• activitynet/
• train_feats.h5
• test_feats.h5
• re_label.mat
• query_feats.h5
• q_label.mat
• fcv/
• fcv_train_feats.h5
• fcv_test_feats.h5
• fcv_test_labels.mat
• hmdb/
• hmdb_train_feats.h5
• hmdb_train_labels.mat
• hmdb_test_feats.h5
• hmdb_test_labels.mat
• ucf/
• ucf_train_feats.h5
• ucf_train_labels.mat
• ucf_test_feats.h5
• ucf_test_labels.mat

configs/

dataset/

inference/

Loss/

model/

optim/

utils/

preprocess.py

train.py

eval.py

requirements.txt

You can downloaded video features from the following Baidu Cloud links and put them into dataset-specific folder under the data/ folder.

Dataset	Video Features	Logs and Checkpoints
ActivityNet	Baidu disk	Baidu disk
FCVID	Baidu disk	Baidu disk
UCF101	Baidu disk	Baidu disk
HMDB51	Baidu disk	Baidu disk

2.3 Train

The training command is as follows:

python train.py --config configs/AutoSSVH/AutoSSVH_<DATASET_NAME>.py --gpu <GPU_ID>

Options:

• <DATASET_NAME>: act, fcv, ucf, hmdb
• <GPU_ID>: specify the gpu id

The logs, model checkpoints will be generated under the logs/ and checkpoint/ folders, respectively.

2.5 Test

We provide the evaluation code for model checkpoints (if exist). The test command is as follows:

python eval.py --configs/AutoSSVH/AutoSSVH_<DATASET_NAME>.py --gpu <GPU_ID>

3. Results

Dataset		Code Length	MAP@5	MAP@20	MAP@40	MAP@60	MAP@80	MAP@100
ActivityNet		16	0.176	0.091	0.055	0.040	0.031	0.026
		32	0.250	0.136	0.082	0.061	0.049	0.038
		64	0.290	0.164	0.098	0.071	0.055	0.045
FCVID		16	0.347	0.256	0.225	0.207	0.193	0.180
		32	0.483	0.336	0.289	0.263	0.244	0.228
		64	0.539	0.381	0.334	0.309	0.290	0.273
UCF101		16	0.423	0.343	0.288	0.244	0.210	0.184
		32	0.519	0.448	0.396	0.353	0.309	0.269
		64	0.570	0.500	0.452	0.413	0.373	0.329
HMDB51		16	0.161	0.113	0.081	0.064	0.053	0.045
		32	0.227	0.163	0.123	0.097	0.081	0.069
		64	0.256	0.175	0.137	0.111	0.090	0.076

4. References

If you find our code useful or use the toolkit in your work, please consider citing:

@inproceedings{Lian25_AutoSSVH,
  author={Lian, Niu and Li, Jun and Wang, Jinpeng and Luo, Ruisheng and Wang, Yaowei and Xia, Shu-Tao and Chen, Bin},
  title={AutoSSVH: Exploring Automated Frame Sampling for Efficient Self-Supervised Video Hashing},
  booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
  year={2025}
}

5. Acknowledgements

This code is based on our previous work ConMH at AAAI'23. We are also grateful for other teams for open-sourcing codes that inspire our work, including S5VH, SSVH, BTH, MCMSH, BerVAE, DKPH, and SHC-IR.

6. Contact

If you have any question, you can raise an issue or email Jinpeng Wang ([email protected]).