landmarks in drone_rolex example

[coder681]

Can you tell us how did you get to create landmarks.json in your example drone_rolex ? (it's not easy to get it manually and also it's less accurate when the views are quite different to do identify landmarks manually)
Thank you

[lcit]

The landmarks.json file contains the position of the drone for all the viewpoints and for multiple time steps. The images in the example are only a capture of at one time point. If I recall correctly, I have manually annotated videos. From videos it is easier to spot it.

[hotohoto]

When you mentioned "the position of the drone," did you mean there's an actual drone visible in the drone_rolex images?

According to the README.md,

The file named landmarks.json contains precise image points for each view that are used to compute fundamental matrices and poses.

Based on this, my understanding is that landmarks.json contains correspondence information, which can potentially be generated automatically using available matchers—such as the one here: https://github.com/Vincentqyw/image-matching-webui.

Please let me know if I misunderstood anything.

[lcit]

Hi @hotohoto, there is indeed a drone in the pictures, it occupies only a few pixels but it is still visible. The frames in that example come from videos that we have annotated to populate the landmarks file. If you want to annotate the position of a moving object from static cameras I suggest using an annotation tool like https://www.cvat.ai/. But it is not necessary to annotate that many landmarks to do the calibration.
And keep in mind that the camera must be time synchronized.

I cannot say much regarding tool to match features across images. They might work or not. If you have wrong correspondences the calibration would not work that well until you filter them out.

[hotohoto]

Thank you for the detailed answer.

Then may I ask—where did all the points in drone_rolex/landmarks.json come from?
It seems like a large number of points, so I’m guessing they weren’t all manually labeled—was there some form of automatic process involved?

[lcit]

The points are the position of the drone in all frames of the original video. We did not provide the video in the example.

If I recall correctly we use a custom detector to locate the drone. But the easiest way to obtain the 2d positions is to use an annotation tool like the one I mentioned. These tools perform interpolation along time and the GUI is smooth so you can annotate thousands of frames fairly quickly.
Regarding the detector, you can use a neural network trained specifically on the object you want to locate or a background subtraction approach. But these need to be prepared and will generate false positive which need to be handled carefully.