SAM3 longer vidoe sequences

[JJjulle]

Hi,

I am trying to develop auto-labeling with vidoes, being about 9000 frames of sequences.
I can't get the streaming working properly, thus my memory get full to quickly.
Further, I am trying to track multiple objects, "person,car,truck".

Does anyone have some good material I can read to or hints on how to tackle this?

[rehan243]

Fwiw, we've been working with SAM on longer video sequences too, around 10k frames, and ran into similar memory issues. We ended up using a combination of PyTorch's gradient checkpointing and a custom dataloader that only loads frames in batches of 100 or so. Turns out, that helped us cut down memory usage by like 70%. We're using SAM with Detectron2 for multi-object tracking - "person, car, truck" is a pretty common use case. What's your current config for SAM looking like? Are you using the original SAM weights or fine-tuning on your own dataset?