COLMAP SphereSFM

[abrahamezzeddine]

Hello,

Thanks for this interstesting work.

I am wondering if this is compatible with COLMAP (SphereSFM), and if so, how to properly implement this as a feature and keypoint matcher? Is it possible?

And also, is it possible to train/fine tune the model yourself?

Thank you!

[jdk9405]

Hi, thanks for your interest in our project!

Yes, integrating our EDM features with COLMAP is part of our ongoing work.
We're currently exploring how to best incorporate these dense features in scene reconstruction.

I’ll need to check internally whether we are able to release the training code.
If it is possible, it may still take some time.

Thanks again for your interest!

[abrahamezzeddine]

Hi, thanks for your interest in our project!

Yes, integrating our EDM features with COLMAP is part of our ongoing work. We're currently exploring how to best incorporate these dense features in scene reconstruction.

I’ll need to check internally whether we are able to release the training code. If it is possible, it may still take some time.

Thanks again for your interest!

I have attempted to do this standalone and then import it and it works ok-ish. But then it starts to loose context and reconstruct rooms into rooms. Not sure why. I am not able to fix the two view geometry which is needed by colmap to fix the correspondence mapping so I have to use the existing two view geometry implementation in sphereSFM (by using feature matching). But I don’t think it can handle dense features so it creates bundles of matches instead (looks very anisotropic rather than isoptropic) unless you provide the two view geometry yourself and import those into colmap.

Below image is from colmap, using EDM for ”sparse” reconstruction but as dense. I could send you the code if you want and hopefully we can manage something?

[jdk9405]

Yes, you have a good point. SphereSfM defines a spherical camera model in two-view geometry and optimization, but I still encountered the same issues when I just integrate their code with our work.

As you know, the model-based COLMAP SfM results depend heavily on the initial prior poses, such as those obtained from two-view geometry.
So, I agree that you have to implement and incorporate additional components for initial pose estimation.

[abrahamezzeddine]

Yes, you have a good point. SphereSfM defines a spherical camera model in two-view geometry and optimization, but I still encountered the same issues when I just integrate their code with our work.

As you know, the model-based COLMAP SfM results depend heavily on the initial prior poses, such as those obtained from two-view geometry. So, I agree that you have to implement and incorporate additional components for initial pose estimation.

I did manage to integrate my own two view geometry independent of sphereSFM’s two view geometry so that I could start reconstructing/mapping immediately without relying on sphereSFM, but I still encountered the same issue.

I can’t really figure out why though… I reviewed the match correspondence for image pairs and the pixel errors is very low. So triangulation should not be very erroneous. Even did proper masking and made sure I was doing bidirectional checks between the image pair warps.

But it seems that in some cases when the matches are anisotropic, this can hurt triangulation / parallax (for example in tight hall spaces) I guess because then I only get points in the far depth from EDM and the points’ position does not change much . This looks like a small but dense bundle of keypoint matches. I suspect some inherent drifting… regular sparse SFM with sphereSFM don’t have this issue.

Quite stuck at the moment, I have no idea how to solve it. 😅 Does the model have to be trained on more datasets to estimate depths better for more diverse images and gain higher confidence on more pixels?

[Golbstein]

Yes, you have a good point. SphereSfM defines a spherical camera model in two-view geometry and optimization, but I still encountered the same issues when I just integrate their code with our work.
As you know, the model-based COLMAP SfM results depend heavily on the initial prior poses, such as those obtained from two-view geometry. So, I agree that you have to implement and incorporate additional components for initial pose estimation.

I did manage to integrate my own two view geometry independent of sphereSFM’s two view geometry so that I could start reconstructing/mapping immediately without relying on sphereSFM, but I still encountered the same issue.

I can’t really figure out why though… I reviewed the match correspondence for image pairs and the pixel errors is very low. So triangulation should not be very erroneous. Even did proper masking and made sure I was doing bidirectional checks between the image pair warps.

But it seems that in some cases when the matches are anisotropic, this can hurt triangulation / parallax (for example in tight hall spaces) I guess because then I only get points in the far depth from EDM and the points’ position does not change much . This looks like a small but dense bundle of keypoint matches. I suspect some inherent drifting… regular sparse SFM with sphereSFM don’t have this issue.

Quite stuck at the moment, I have no idea how to solve it. 😅 Does the model have to be trained on more datasets to estimate depths better for more diverse images and gain higher confidence on more pixels?

Hey! I've implemented feature tracks based on this code and am currently working on integrating it with OpenMVG.

I used an "aggregate and merge" approach — aggregating all keypoints from all image pairs, then clustering them into frozen canonical keypoints. I also performed bidirectional consistency checks using bearing vectors instead of pixel coordinates.

Additionally, I filtered out matches with certainty below 0.8. I'd be curious to hear your thoughts on that threshold — do you think it's too strict or reasonable?

btw, have you made any progress on your end, or are you still running into issues? Would be great to sync up and compare notes!

[abrahamezzeddine]

@Golbstein

Here is how it's going so far... only using EDM to estimate dense keypoints, then I reconstruct it myself using depth maps to guide the reconstruction and a diverse set of algorithms.

each red circle is a camera position, so I have implemented this totally standalone in python, not relying on any existing SFM-programs. But still work in progress, a lot to be done here.

[jdk9405]

We plan to release the SfM code with EDM features. Please check out this project for more details!

https://jdk9405.github.io/IM360/

[abrahamezzeddine]

We plan to release the SfM code with EDM features. Please check out this project for more details!

https://jdk9405.github.io/IM360/

Sounds great!!! 😁 Thanks for putting your effort into this. Really helps the community.