Training Issues with Medium/Slow Configurations on Maternal Behavior Videos

[HOLGUINsantiago]

Hi,

I’ve been working with DeepEthogram for the past few weeks to automate the behavioral labeling of maternal care videos. We're studying several categories of behaviors, including:

• Maternal-pup interactions: grooming pups, carrying pups, kicking, nursing
• Maternal self-behaviors: self-grooming, eating/drinking
• Positional states: on-nest and off-nest
• Nest-building activity

We work with two distinct experimental conditions that significantly affect the cage content and visual environment.

We're very really happy with DeepEthogram’s performance overall (F1 scores between 0.55 and 0.98), but we’ve encountered difficulties with behaviors that are less frequent in the dataset. Despite having at least 13,274 labeled frames per class, training with more thorough configurations has been problematic:

1. Medium and slow configurations result in significantly worse F1 scores and overall performance compared to the fast setting.
2. Separating interaction behaviors and positional clues into separate classifiers leads to similarly poor results.

We suspect that the strong performance of our initial “fast” model may be due to incremental training — it was fine-tuned over time as we added more annotated data, possibly transferring knowledge from model “m-1” to model “m.” In contrast, our newer models are trained from scratch, which might explain the performance drop.

Would you have any recommendations for replicating this incremental training effect or for improving model performance in the slower configurations (e.g., by extending training duration or using weight initialization)?

Thank you very much for your time and for developing such a powerful tool.