Support multi-animal pretrained models in DataJoint DeepLabCut Element

[maryapp]

Feature Request

Problem

The DeepLabCut software released the model zoo in 2020, which provides access to pretrained models. Last year, we released the SuperAnimal family of models (see Ye et al. 2024 Nature Communications, which are good foundational models for pose estimation across mice and/or quadrupeds. We would like the Element to support video inference with these models.

Specifically, we would like to follow the suggested workflow from Mathis et al. 2020 Neuron:

Source: https://www.sciencedirect.com/science/article/pii/S0896627320307170

Here, the user should be able to train/deploy OR grab a model and deploy. We are excited to help contribute this feature!

Requirements

• Select & ingest multi-animal SuperAnimal models.
• Minimal docs + example notebook.

Implementation

• new SuperAnimal_Model schema
• new general model table that is the parent of existing Model and SuperAnimalModel tables
• the results Compute table (output of deeplabcut.video_inference_superanimal function call)

[esutlie]

Hi Mary, this is a great idea! We definitely want to add support for pretrained models and it would be awesome to have the SuperAnimal models available by default.

We’d love to build this with you, so let’s discuss the implementation details once our team has had a chance to review your draft PR. It would also be great if you could help with testing once we’re close to merging.

We’ll review and get back to you within the next two weeks. I look forward to working with you on this!