error:Less than 2 images defined. when running "Step-by-step example (individual images)"

[kakghiroshi]

Hi!
I was recently following the steps in the" Step-by-step example (individual images) "to run the code.I got some problem in step

external SfM tool and external surface reconstruction tool

1、When i download the official SfM (Structure-from-Motion) reconstruction containing the DSLR images and the cube map images you posed in

An SfM (Structure-from-Motion) reconstruction containing the DSLR images and the cube map images must be created using an external tool. We provide an example SfM reconstruction here (unzip into the same directory as the input data archives).

the TX、TY and TZ in image.txt is

-inf -inf inf

I estimate the scaled factor using
mkdir sparse_reconstruction_scaled ${PIPELINE_PATH}/SfMScaleEstimator -s sparse_reconstruction -si . -i scan_clean -o sparse_reconstruction_scaled --cube_map_face_camera_id 0 .
when i execute the command
${PIPELINE_PATH}/ImageRegistrator \ --scan_alignment_path scan_clean/scan_alignment.mlp \ --occlusion_mesh_path surface_reconstruction/surface.ply \ --occlusion_splats_path surface_reconstruction/splats.ply \ --multi_res_point_cloud_directory_path multi_res_point_cloud_cache \ --image_base_path . \ --state_path sparse_reconstruction_scaled/colmap_model \ --output_folder_path dslr_calibration_jpg \ --observations_cache_path observations_cache \ --camera_ids_to_ignore 0
the error"Less than 2 images defined"occured.（PS： The input data of this code incluing sparse_reconstruction_scaled/colmap_model ）

I found that the problem is with function ReadColmapImages in the file colmap_model.cc.

The information was successfully read into the string line.But looks like the function std::istringstream can not recognized

-inf -inf inf

in line,so the data after the first

inf （Either positive or negative）

can't be read into
>> new_image->image_T_global.data()[4] >> new_image->image_T_global.data()[5] >> new_image->image_T_global.data()[6] >> new_image->camera_id >> new_image->file_path;respectively.

By the way,I am running this project on ubuntu 20.04.The environment is built according to the instructions in dockfile.

Any suggestion about how to deal with this problem would be welcome.

I also want to know how did you get two camera model and

-inf -inf inf

by using COLMAP.

I would really appreciate it if you could answer my question！

[ClementPinard]

Hello,

it seems the reader is not working properly. image_id should obviously be only an integer, so I suspect that expected and actual orders are different.

Are you able to run COLMAP on your computer ? Can you try the same thing with a SfM model created from scratch instead of the provided one ? The given model dates from 2017 so it's possible the file format has changed since.

[kakghiroshi]

Thanks for your quick reply! Of course I can run COLMAP on my computer, but I don't know how to import the intrinsics parameters of the two cameras into COLMAP, and I can't set the camera id to 0. It seems that the camera id must be greater than 0.This is the model I created：

As you have mention in your paper

T. Schöps, J. L. Schönberger, S. Galliani, T. Sattler, K. Schindler, M. Pollefeys, A. Geiger, "A Multi-View Stereo Benchmark with High-Resolution Images and Multi-Camera Videos", Conference on Computer Vision and Pattern Recognition (CVPR), 2017.

I include rendered cube map images for each scan position into the SfM reconstruction,so there are 35 images in total（23 original and 12 rendered cube map）.
You can see that only 28 images are output successfully while the provided one output 33 images,maybe there are other operations that need to be done in between?
I would be very grateful if you could give the corresponding steps.
According to the model I created I changed the code from ${PIPELINE_PATH}/SfMScaleEstimator -s sparse_reconstruction -si . -i scan_clean -o sparse_reconstruction_scaled --cube_map_face_camera_id 0 to ${PIPELINE_PATH}/SfMScaleEstimator -s sparse_reconstruction -si . -i scan_clean -o sparse_reconstruction_scaled --cube_map_face_camera_id 2.

Then，error occurred.

Do you have any suggestion?

[ClementPinard]

You can see that only 28 images are output successfully while the provided one output 33 images,maybe there are other operations that need to be done in between?

It's not unusual to have missing images when you run COLMAP without refinement. Here is a set of advices to add more images to the model :
https://colmap.github.io/faq.html (see Increase number of matches
I do concede that it can be somehow empirical, just test different parameters and see how it goes. The good news is that there are very few images so the trial and error should not be too painful.

Then，error occurred. Do you have any suggestion?

It's strange that the intrinsics file was not created. See this line that is normally run when launching cubeMapRenderer :
https://github.com/ETH3D/dataset-pipeline/blob/master/src/exe/cube_map_renderer.cc#L146

Can you specify the options you used for this command ?

[kakghiroshi]

Can you specify the options you used for this command ?

Sure,I executed the exact same command as you provided：

${PIPELINE_PATH}/CubeMapRenderer -c scan_clean/scan1.ply -o cube_maps/scan1.ply --size 2048 ${PIPELINE_PATH}/CubeMapRenderer -c scan_clean/scan2.ply -o cube_maps/scan2.ply --size 2048.

Actually the intrinsics file has been created, but the code can't read it.

At the same time, if I used my own created model, should the ignored camera id be modified according to my model, for example ignored_camera_id set to 2.

[ClementPinard]

It's indeed very strange. The error you get is definitely a problem of the file not being open (it still hasn't even checked if it is readable), meaning the file appears to be missing.

One shot in the dark : try to replace . by the actual absolute path in your SfMScaleEstimator command.

${PIPELINE_PATH}/SfMScaleEstimator -s sparse_reconstruction -si ${PWD} -i scan_clean -o sparse_reconstruction_scaled --cube_map_face_camera_id 2

[kakghiroshi]

Thanks for your kindly help!
Problem

Actually the intrinsics file has been created, but the code can't read it.

is solved.I still follow the Step-by-step example (individual images) to run the code.

At the same time,I found that although the translation vector (TX, TY, TZ) is numeric in my model.when output after step SfMScaleEstimator it become

inf

again.

As you can see,

dataset-pipeline/src/io/colmap_model.cc

Lines 130 to 134 in dc4a106

	if(opt::GlobalParameters().scale_factor != 0){
	new_image->image_T_global.translation() *= opt::GlobalParameters().scale_factor;
	}else{
	LOG(ERROR) << "Please load point clouds before images";
	}

shows that if scale_factor=0,errorPlease load point clouds before images occured.
I do get this error:

Code

dataset-pipeline/src/io/colmap_model.cc

Line 166 in dc4a106

float scale = opt::GlobalParameters().scale_factor;

and

dataset-pipeline/src/io/colmap_model.cc

Line 172 in dc4a106

<< " " << colmap_image.image_T_global.data()[4] / scale

shows that if scaled_factor equal to 0,code will definitely output

inf

From

dataset-pipeline/src/opt/parameters.h

Line 64 in dc4a106

scale_factor = 0; // 0 means that it will be replaced by inverse of first point cloud transform scale

,the default value of scaled_factor is 0,it should be replaced by inverse of first point cloud transform scale. So maybe the point cloud didn't load correctly?
Do you have any suggestion?

[ClementPinard]

Just did some tests on my own using the docker image, and did not stumble upon your problems. Interestingly enough, there was another scale factor problem.

Try to cheeckout #15 and see how it goes, maybe it will solve your problem ?

Otherwise, the tests pass on my side so not sure what is your problem, maybe there are some problems with the std lib and your gcc version.

[kakghiroshi]

I have set the global factor to 1 as you suggested #15.

Running a test with my own model using the docker image

A Dockerfile is given to build everything from scratch in ubuntu 20.04.

Still in ImageRegistrator,error

occures at

dataset-pipeline/src/io/colmap_model.cc

Lines 784 to 786 in dc4a106

	void CheckEqualHelper(int a, int b) {
	CHECK_EQ(a, b);
	}

When using the model provided by

We provide an example SfM reconstruction here (unzip into the same directory as the input data archives).

,this problem wouldn't happen.（But I still don't know if the code will work as it's still iterating）
This problem arises very strangely because my model is also generated with COLMAP.What is the key to making aequal to b？
My models are listed below：
cameras.txt
images.txt
points3D.txt

[ClementPinard]

(Sorry the issue got automatically fixed, wasn't intentional, I reopened it)

[ClementPinard]

Glad to see we are making progress ! :)

See Colmap's thin prism fisheye definition here :
https://github.com/colmap/colmap/blob/master/src/base/camera_models.h#L349

As you can see, it's 12 parameters. ETH3D checks that the camera model described in the cameras.txt matches the number of floats in the line, hence the given assertion. For some reason, ETH3D reads 13 parameters which then fails because because it's not 12.

See here for how the camera line is read :

dataset-pipeline/src/io/colmap_model.cc

Line 53 in dc4a106

bool ReadColmapCameras(const std::string& cameras_txt_path,

It seems to me the cameras.txt you provided is fine, so something goes wrong during the reading.

I will do tests on my own, might be worth it trying to log what are these 13 parameters to understand where things went wrong.

[kakghiroshi]

Thanks for your work！:thumbsup:
I have found the cause of the problem.I firstly using COLMAP-3.7-windows-no-cuda.zip in windows,I guess Windows have different output format compare with Linux(build from source code) version.Specifically,the escape characters they use are different, so the model generated by windows is incorrectly recognized in linux,like:

The actual size of the parameter line is 55,The code does recognize that the string size is 56.I guess the extra string contains nothing, so it is 0.Using the model created by Linux version COLMAP,the problem is fixed:

Stil running Step-by-step example 😤 ,still in ImageRegistrator using code

${PIPELINE_PATH}/ImageRegistrator
--scan_alignment_path scan_clean/scan_alignment.mlp
--occlusion_mesh_path surface_reconstruction/surface.ply
--occlusion_splats_path surface_reconstruction/splats.ply
--multi_res_point_cloud_directory_path multi_res_point_cloud_cache
--image_base_path .
--state_path sparse_reconstruction_scaled/colmap_model
--output_folder_path dslr_calibration_jpg
--observations_cache_path observations_cache
--camera_ids_to_ignore 2

Problem

occurred.I try to replace --image_base_path . \ with --image_base_path dslr_images \,it doesn't work.

The code I added:

Any suggestion would be needed.Thanks again!

[kakghiroshi]

Hi！
Recently I am trying to apply your code to my project.However, the difference is that the point cloud we obtain does not contain the property of color.The point cloud is densified by slam and stitched together.In the step CubeMapRenderer ,cube map face images are rendered from the laser scans with color.I wonder if the property of intensity can be used to render cube map face images instead of color(for our point cloud don't have the property of color) in step CubeMapRenderer.
Of course, the code needs to be modified to achieve intensity image rendering.
Can you give me some advice?Thanks! 😊