SFM: Known intrinsic parameters

[hcjghr]

Hi,

I was looking at the code but I was not able to find any way to introduce known intrinsic camera parameters with the images. Did I just overlooked it, or for now there isn't any efficient way to do it before computing SFM reconstruction?

Thanks

[simonfuhrmann]

Currently, there is indeed no obvious way to introduce known camera intrinsics. The focal length is obtained from the EXIF tags, and principal point is set to the image center. One would have to inject the known focal length into EXIF and use --fixed-intrinsics, but that is a crappy solution.

I have to look into what must be done to initialize the intrinsics from the views. In combination with --fixed-intrinsics, this should do the job. Then it's up to you to put your known intrinsics into the views before SfM reconstruction.

[simonfuhrmann]

This feature request raised a few design issues. Most importantly, we need to normalize feature coordinates which is a larger change in the SfM library. Also, the prebundle.sfm file might change to a matching.sfm file. This suggests that we implement this change before the release of MVE2.

[simonfuhrmann]

Prerequisite issue #126.

[simonfuhrmann]

A feature --intrinsics-from-views has been added to sfmrecon obtain intrinsic parameters from the views. Currently focal length is the only intrinsic parameter that is supported (camera.focal_length in the meta.ini file).

Is this already useful for you in the current state?

[Sudhir1990]

Hello,

I am looking for a solution to include known intrinsics(all intrinsics along with focal length).

Is it possible to include known intrinsics other than focal length as well in the EXIF info? If yes, what should be the format of exif info? Also can we include approximate positions?

Many thanks!

[simonfuhrmann]

Only the focal length can be provided in the EXIF info. A precise focal length can be provided in the meta.ini file. Radial distortion, however, can not be provided at this time. We are working on that.

[tobycollins]

A solution to handle arbitrary intrinsics could be as follows. You warp each image using its calibration intrinsics, so that the warped image has a canonical intrinsic matrix of [f, 0, imageWidth/2; 0; f; imgHeight/2; 0, 0,1]. Then you just plug f into the EXIF tag, and it's done.

[simonfuhrmann]

This will, however, still optimize the intrinsics if --fixed-intrinsics is not specified. Also, unfortuantely, EXIF only supports integer values for the focal length, IIRC.

[tobycollins]

Hi - yes I assumed --fixed-intrinsics would be set :-) Regarding the fact that EXIF only supports integer values, it's not really a problem. The canonical intrinsic matrix can have an arbitrary f (and so can be set to an integer). This could be done by rounding the original intrinsic matrix's fx value.

So the workflow could look something like:

1. Undistort the input image to eliminate effects of lens distortion. This can be done with e.g. opencv's undistort method. Let's call this undistorted image I. This has an intrinsic matrix K (which is the same as the original input image).
2. Define a canonical intrinsic matrix K2 = [f,0,w/2;0,f,h/2,0,0,1] with f = round(K(0,0))
3. Warp I using a homography H, so that the warped image's intrinsic matrix is K2. To do this you use H = K2*inv(K). Let's call the warped image I2.
4. Set the focal length in the EXIF tag to f and specify --fixed-intrinsics, then perform SfM on the warped images.

The nice thing about this is that beyond some image pre-processing there would be no significant code/design changes. One thing to point out is that when you undistort an image and warp it with a homography there can be visible borders that correspond to the perimeter of the original image. These borders may produce spurious features and it could be necessary to remove them.

Hope this helps.

[simonfuhrmann]

That's an interesting approach. If you set f slightly larger than initially, the homography H will essentially crop a little bit the image thus preventing visible borders.

[drbuckingham]

I added camera.focal_length = 20 to the meta.ini file of all the views in the scene, but every view reported "Warning: View X has zero focal length. Using fallback.". I am new to the process, so perhaps I am missing something obvious to others.

[simonfuhrmann]

First, focal_length = 20 doesn't sound right. The focal length is given in normalized format, e.g., if you have a 70mm lens on a 36mm sensor, you'd pass 1.9444. So you need to know your sensor size, which is the whole point of MVE having a camera database. (If your camera is missing from that database, feel free to add it or provide an EXIF dump.)

Second, you also have to pass --intrinsics-from-views to tell MVE to use the values you provided in the meta.ini file. Otherwise it will try to obtain the information from EXIF.