Skeleton design with modules

[hawkinso]

Hi! I have tried to look at previous posts and discussions regarding skeleton design, and after applying them, I still can't get a few points to ever be predicted.

For context, I have a skeleton (shown below) where I have a few modules for each of the fin types (i.e., dorsal fin points connect to a parent dorsal point and that point goes to the eye). I tried a few skeleton arrangements for the caudal fin/tail fin, but no matter what, the dorsal and ventral lobes are never detected. Instead, what I would consider the most difficult points to detect (the peduncle itself and the middle part of the fin) are detected every time. The idea behind having the modules was so when I use the same skeleton in a set of videos that have a different view, the non visible modules would just be completely ignored. I saw another post where the solution was just to move the anchor point more centrally to the points that were not showing up, but I am not sure if I can have more than one anchor point (since the one I have now for the rest of the modules seems to work well).

I have tried making the input scaling smaller while keeping the max stride large, and that has not helped (maybe I am missing some information on what that would help with in this situation). The training parameters are included in this post as well. I am using single instance configuration! I currently have 6 videos (but this will eventually be a project of upwards of 50 videos), and each video has a minimum of 10 labeled frames.

I appreciate any guidance you could provide on this issue!

[github-actions[bot]]

Thank you for starting a new discussion!

We appreciate your input and will review it soon.

Warning

A friendly reminder that this is a public forum. Please be cautious when clicking links, downloading files, or running scripts posted by others.

• Always verify the credibility of links and code.
• Avoid running scripts or installing files from untrusted sources.
• If you're unsure, ask for clarification before proceeding.

Stay safe and happy SLEAPing!

Best regards,
The Team

[eberrigan]

Hi @hawkinso,

• Please go through your labels and make sure those nodes or any others are not unintentionally toggled as invisible (grey nodes). If they are, the model will learn to make these nodes invisible. Right-clicking the node toggles them from visible to invisible and vice-versa.
• Your receptive field size looks a little large. The rule of thumb is to have a receptive field size approximately equal to your animal. Please try increasing the input scaling so that the preview of your receptive field (blue box) is approximately the size of the fish.
• What are the lengths and widths of your fish and tank in pixels? I would like to know what the ratio of fish to tank is approximately. Single instance model is probably fine. See guide Troubleshooting Workflows.
  • Please note that anchors are only beneficial for top-down models and part-affinity are only calculated for bottom-up models (edge definition does not influence model performance for single instance models).
• Since you have many nodes in your skeleton, you can increase the number of filters. If this does not work you can even try training models for each component of your skeleton separately.
• We have also noticed that some difficult skeletons do much better after training once and then resuming training from this trained model.
• You can try online mining. This dynamically reweights the emphasis on hard nodes during training. We recommend doing this with resume training because it can destabilize to focus on hard nodes when starting from a model that can do easy nodes.