DINOv3 Integration and Multi-Task Architecture Discussion

[aliejabbari]

hey !
Quick question: Any plans to check out DINOv3 as the new backbone?

Also, have you thought about running RoMa and segmentation/detection (like SAM) off of one shared backbone? It could make matching way cleaner if it was "grounded" by the detected regions.

Thanks for the great release!

[Parskatt]

Hi, for dinov3 you may want to check out roma v2: https://github.com/Parskatt/RoMaV2

[Parskatt]

As for segmentation, there are some works in that direction, where they use multicrops to consttain matches within semantic. Dont remember the names.

It works well for indoor, but im skeptical about cluttered outdoor scenes.

[aliejabbari]

Hey, thanks for the pointers and for releasing RoMa v2 it looks great, really nice work on pushing accuracy while keeping memory and inference time reasonable.

I’m exploring whether a single shared backbone could both (i) provide semantic / concept level segmentation and (ii) support RoMa style dense matching between semantically similar regions, to better ground the matches in segmented areas.