Using SLEAP on a directory of images (predefined frames) rather than a video

[Mattshanevdberg]

Hi please can you provide some help.

I am benchmarking different animal pose estimation algorithms against each other. I need to compare pose estimation on the specific frames that I have already extracted from my videos. Is it possible to train a model and/or run inference on a directory containing extracted frames rather than on the original video?

Thank you so much for your time

[github-actions[bot]]

Thank you for starting a new discussion!

We appreciate your input and will review it soon.

Warning

A friendly reminder that this is a public forum. Please be cautious when clicking links, downloading files, or running scripts posted by others.

• Always verify the credibility of links and code.
• Avoid running scripts or installing files from untrusted sources.
• If you're unsure, ask for clarification before proceeding.

Stay safe and happy SLEAPing!

Best regards,
The Team

[eberrigan]

Hi @Mattshanevdberg,

Yes sleap-io has functionality for reading and writing labels in a variety of formats. What format are your annotations in and what format would you like them to be saved to?

Thanks,

Elizabeth

[Mattshanevdberg]

Hi Elizabeth,

Thank you so much for you quick response, appreciate it!

I cloned this sleap repository. Would it have the same functionality within it? Or is sleap-io an additional functionality repository?

So I have some annotations in the DLC format that I have managed to convert. But the rest of the data still needs annotation and I would like to use SLEAP to assist with that. The only thing is, I already have specific frames that were extracted from the videos (and I need to compare the performance on those exact frames). So is there a function that would allow me to directly perform pose estimation on a set of frames rather than a video? (pass the algorithm frames as apposed to a video input). I imagine there is, but I can't seem to find where the actual frames are stored.

Thanks so much!

[eberrigan]

Hi @Mattshanevdberg,

There are a few way to do this.

• You can use sleap-track and specify the --frames.
• You can make a new .slp file using sleap-io. You will read your labels.slp using sleap_io.load_slp which returns a Labels object which has labeled_frames as an attribute.

import sleap_io as sio
from pathlib import Path

# Define paths
labels_path = Path("/your/labels.slp")
new_labels_path = Path("specify/path/to/save/new/labels.slp")

# Load the SLP file
labels = sio.load_slp(labels_path)
labeled_frames_list = labels.labeled_frames

# Specify desired frames as integer values
wanted_frames = [1, 5, 10]  # Example: list of desired frame indices

# Filter frames
specific_frames = [frame for frame in labeled_frames_list if frame.frame_idx in wanted_frames]

# Create new Labels object with the selected frames
new_labels = sio.Labels(labeled_frames=specific_frames)

# Save the new Labels object to a file
sio.save_slp(new_labels, new_labels_path)

• Something similar can be done with the SLEAP API if you would like to use one environment for everything since sleap-io does not have any functionality related to labeling, training, or inference.

Please let me know if you have any questions!

Thanks,

Elizabeth

[Mattshanevdberg]

Hi Elizabeth,

Thank you so much for this. Yes, I can see how this would work. I had to jump on a more urgent matter and am returning to this topic this week so I will let you know how it goes.

I did see that you can use sleap-track (!sleap-track "/path/to/image_folder" -m "/path/to/centriod_model_folder" -m /path/to/center_instance_model_folder") and direct it to a folder containing images and perform inference on the images within that folder. This gave me an output. I have not inspected the results yet, but it seems to be what I would be looking for.

Thank you again for the assistance! It is always appreciated. (I will close this by the end of the week, once I have checked the output)

[eberrigan]

Please see #1979 as well.