Skip to content

Add First Pass Tracking Functionality#33

Open
emdavis02 wants to merge 15 commits into
mainfrom
elise/first_pass_tracking_classes
Open

Add First Pass Tracking Functionality#33
emdavis02 wants to merge 15 commits into
mainfrom
elise/first_pass_tracking_classes

Conversation

@emdavis02

@emdavis02 emdavis02 commented Jul 2, 2025

Copy link
Copy Markdown
Contributor

PR Description

The sleap-mot repo currently supports tracking pose-estimated videos either:

  1. From scratch (no prior identity information), or
  2. With initial tracklets produced from first-pass tracking.

First-pass tracking uses visible features or external signals to assign partial identities (tracklets) to animals when those features are available. For example:

  • Visual features like fur color can distinguish animals when they're unobstructed.
  • RFID pings can be matched to individuals when a signal is detected.

Previously, these feature-based tracking methods were implemented in scattered Jupyter notebooks. The goal of this PR is to incorporate them directly into the sleap-mot repository, enabling it to function as a complete identity tracking pipeline, from raw pose data to globally consistent tracks.


Key Changes

  • Introduced a general FeatureTracker base class
    This abstract base class defines shared functionality and interfaces for all feature-based tracking approaches. It includes:
    • Utility functions for tracklet management
    • A common structure for integrating with the broader tracking pipeline
    • An abstract method track() that must be implemented differently in each subclass depending on the feature type

Implemented subclasses:

  • RFIDFeatureTracker
    This subclass demonstrates how specific feature-based methods (in this case, RFID-based) can be cleanly integrated using the inherited structure and shared utilities from FeatureTracker.

  • FurColorFeatureTracker

  • Test functions will be written to test the functionality of these new classes in tandem with the tracker class (second step tracking)


Outcome

This PR formalizes and integrates feature-based first-pass tracking into the core sleap-mot package. It lays the groundwork for a modular, extensible identity tracking system, where multiple feature types (e.g., color, RFID, embeddings) can be easily added and composed.

@codecov

codecov Bot commented Jul 2, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 2.48500% with 1138 lines in your changes missing coverage. Please review.
✅ Project coverage is 20.81%. Comparing base (2cb2db0) to head (4dc8d41).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
sleap_mot/feature_tracker.py 0.00% 994 Missing ⚠️
sleap_mot/tracker.py 26.19% 62 Missing ⚠️
sleap_mot/utils.py 12.50% 49 Missing ⚠️
sleap_mot/eval.py 0.00% 33 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (2cb2db0) and HEAD (4dc8d41). Click for more details.

HEAD has 2 uploads less than BASE
Flag BASE (2cb2db0) HEAD (4dc8d41)
5 3
Additional details and impacted files
@@             Coverage Diff             @@
##             main      #33       +/-   ##
===========================================
- Coverage   44.73%   20.81%   -23.92%     
===========================================
  Files           8        9        +1     
  Lines         930     2066     +1136     
===========================================
+ Hits          416      430       +14     
- Misses        514     1636     +1122     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread sleap_mot/feature_tracker.py Outdated
@@ -188,12 +192,20 @@ def extract_tracking_data(self, labels):
# Initialize a dictionary for this row with 'frame_idx' and 'track'
row = {"frame_idx": frame_data.frame_idx, "track": inst.track.name}
for point in inst.points:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could do something like:

cols = [*[f"{node_name}.x", f"{node_name}.y", f"{node_name}.score"] for node_name in skeleton.node_names]

pd.DataFrame(inst.numpy(scores=True).reshape(-1), columns=cols)

…tracker classes

- updated environment
- Expanded docstrings in the FeatureTracker class to include detailed argument descriptions.
- Implemented force matching for untracked instances in the Tracker class, including methods for resolving track IDs and finding closest global track instances.
- Improved handling of untracked instances and global tracks during tracking process.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants