Zero-Shot Multi-Animal Tracking in the Wild

📝 Overview

TLDR. We modify SAM2MOT for multi-animal tracking. Due to our adaptive detection threshold, no hyperparameter adaptations between different datasets are necessary. No adaptation, no finetuning, just throw it on your data (fully zero-shot).

📰 News

• 20.10.2025: A preliminary version of our paper is released on arxiv.

🚀 Getting Started

TLDR. Follow the installation instructions, add your dataset in dancetrack format and run this command:

python run.py --dataset <your dataset name>

• See INSTALL.md for installation instructions.
• See DATASET.md for dataset downloading and preprocessing.
• See DETECTION.md for information about supported detection models and loading detections.
• See TRACKING.md for how to run tracking on a dataset.

📊 Results

Dataset	HOTA↑	DetA↑	AssA↑	DetRe↑	LocA↑	MOTA↑	IDF1↑	IDSW↓
ChimpAct	58.6	49.8	70.1	57.3	83.4	48.6	66.7	32
BFT	74.8	72.2	77.7	80.5	87.8	81.8	88.4	51
AnimalTrack	58.0	52.7	65.2	63.8	81.1	58.9	72.0	442
GMOT-40-Animal	62.4	57.2	69.2	67.2	80.1	64.7	77.4	496

📁 Supported Datasets

The following datasets are supported out of the box. More/custom datasets can be easily added (See DATASET.md).

• Animals
• • ChimpAct: Chimpanzees in the Leibzig Zoo.
• • Bird Flock Tracking (BFT): Different bird species in diverse Envionments.
• • AnimalTrack: Diverse selection of 10 common animal categories.
• • GMOT-40-Animal): 4 different animal categories in crowded scenarios.
• • PanAf500: Camer trap videos of chimpanzees in their natural environment.
• Persons
• • DanceTrack: Dancers with unifrom appearance and diverse motion.
• • SportsMot: Athletes in diverse sport scenes.
• Vehicles
• • UAVDT: Vehicles in complex scenes filmed with drones.
• • BDD100k: Driving videos with multiple object classes.

🛠️ Supported Detectors

The following detectors are supported out of the box. More/custom detectors can be easily added (See DETECTION.md).

• Huggingface
• • e.g. OWLv2, Grounding Dino, LLMDet
• mmDetection
• • e.g. Grounding Dino
• Already existing detections

🤝 Acknowledgements

This project is build upon SAM2, SAM2MOT and TrackEval. We thank the authors for their amazing work.

📚 Citation

If you think this project is helpful, please feel free to leave a ⭐ and cite our paper:

@misc{meier2025zeroshotmultianimaltrackingwild,
      title={Zero-Shot Multi-Animal Tracking in the Wild}, 
      author={Jan Frederik Meier and Timo Lüddecke},
      year={2025},
      eprint={2511.02591},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2511.02591}, 
}