Error calibrating multiple camera - Did not converge in maxIterations

[jsreveal]

Dear all,

I am having issues calibrating a setup with two cameras, with error: Did not converge in maxIterations... restarting...

Output:

Initializing cam0:
	Camera model:	  pinhole-equi
	Dataset:          calibr4.bag
	Topic:            /cam0/image_raw
	Number of images: 431
Extracting calibration target corners
  Extracted corners for 223 images (of 431 images)                              
	Projection initialized to: [723.18613989 723.16578287 661.1724854  325.56671198]
	Distortion initialized to: [-0.0317777  -0.01434668 -0.02453107  0.02217757]
Initializing cam1:
	Camera model:	  pinhole-equi
	Dataset:          calibr4.bag
	Topic:            /cam1/image_raw
	Number of images: 431
Extracting calibration target corners
  Extracted corners for 278 images (of 431 images)                              
	Projection initialized to: [703.11420155 703.23239151 659.16861833 382.55004172]
	Distortion initialized to: [-0.00519049 -0.04299709  0.02997729 -0.01339708]
initializing initial guesses
	 initializing camera pair (0,1)...  
initialized baseline between cam0 and cam1 to:
[[ 0.98553942 -0.03046893 -0.1666844   0.12364618]
 [ 0.02658238  0.99932136 -0.02549887 -0.01188539]
 [ 0.16734821  0.02069928  0.98568053 -0.0074947 ]
 [ 0.          0.          0.          1.        ]]
initialized cam0 to:
	 projection cam0: [1033.51355326 1008.11780521  596.98674151  395.81607817]
	 distortion cam0: [ 0.03322519 -1.53655711  2.51260886 -1.31788827]
initialized cam1 to:
	 projection cam1: [960.6536291  977.20984921 616.73353001 506.59829856]
	 distortion cam1: [-0.01077812 -0.51467782  0.45530982 -0.13393893]
initializing calibrator
starting calibration...
  Progress 6 / 295 	 Time remaining: 50s                  [ERROR] [1729682712.625799]: Did not converge in maxIterations... restarting...
[ WARN] [1729682712.634584]: Optimization diverged possibly due to a bad initialization. (Do the models fit the lenses well?)
[ WARN] [1729682712.644695]: Restarting for a new attempt...

If I ran calibration on a single camera in the same bag, it manages to finish the calibration with success for each camera.

cam0 calibration results

Calibration results 
====================
Camera-system parameters:
cam0 (/cam0/image_raw):
    type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>
    distortion: [ 0.00187256 -0.03032581  0.00631689 -0.00196265] +- [0.00967692 0.02235402 0.03466247 0.01822428]
    projection: [687.99995267 687.60616451 659.84565099 325.71475583] +- [8.37495216 8.43195714 0.63865365 0.75649313]
    reprojection error: [-0.000005, 0.000002] +- [0.684067, 0.450114]



Target configuration
====================

  Type: aprilgrid
  Tags: 
    Rows: 6
    Cols: 5
    Size: 0.0275 [m]
    Spacing 0.00825 [m]

cam1 calibration results

Camera-system parameters:
cam0 (/cam1/image_raw):
    type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>
    distortion: [ 0.06289355 -0.10122094  0.0895506  -0.03688238] +- [0.011619   0.0154553  0.01998371 0.00857068]
    projection: [645.9490954  645.97843407 658.83766247 381.42173618] +- [11.8508694  11.87081626  0.65490459  0.77522154]
    reprojection error: [0.000003, 0.000001] +- [0.737018, 0.456293]


Target configuration
====================

  Type: aprilgrid
  Tags: 
    Rows: 6
    Cols: 5
    Size: 0.0275 [m]
    Spacing 0.00825 [m]

Can anyone help or guide me here on what I can do, or should modify.

[FletcherFT]

I have had exactly the same issue. I have been able to link it to L245 of kalibr_calibrate_cameras. If you set the epsilon of the SVD solver to be higher (in my case 1e-3), then the solver manages to reach convergence under the maxIterations parameter.

However, this probably will affect the quality of the output calibration. My output seems reasonable and agrees more or less with other methods, like opencv, so I'll accept that.

Could be worth bringing some of these "fine-tuning" parameters up to the CLI.