Remove tied weight keys Sam2Video #42840
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| _tied_weights_keys = { | ||
| "prompt_encoder.shared_embedding.positional_embedding": "shared_image_embedding.positional_embedding" | ||
| } | ||
| # need to be ignored, as it's a buffer and will not be correctly detected as tied weight | ||
| _keys_to_ignore_on_load_missing = ["prompt_encoder.shared_embedding.positional_embedding"] |
There was a problem hiding this comment.
Is it really not needed at all anymore? We could otherwise very easily allow buffers to be tied (but a bit weird)
There was a problem hiding this comment.
Everything seems to work just fine without it 🤷, no warnings either
There was a problem hiding this comment.
Well, for sure you won't have warnings about tied weights if we never tie them haha - did you make sure that both weights correctly live inside ALL main checkpoints? If it's the case, then yes probably they were never supposed to be tied
There was a problem hiding this comment.
Yes all is working fine, the tying was also removed for sam/sam2, we just forgot about this one I guess
|
[For maintainers] Suggested jobs to run (before merge) run-slow: edgetam_video, sam2_video, sam3_tracker_video |
3 participants
Not necessary anymore, and was crashing the model.
Fixes #42837