MinimalSLAM: Visual SLAM Pipeline

This project implements a minimal SLAM pipeline using OpenCV and Ceres Solver. It reconstructs 3D points and estimates camera poses from a sequence of images. The reconstructed point cloud can be visualized using Open3D via the included view.py script.

The application is implemented in C++ for performance, with Python scripts for visualization. The pipeline includes feature detection, matching, camera pose estimation, loop closure, pose graph optimization and 3D points triangulation.

Note: The algorithm assumes that the images were collected in a loop and that the last image overlaps with the first one for loop closure.

---

Features

• Feature detection and matching using SIFT and FLANN
• Camera pose estimation via essential matrix decomposition
• Loop closure and pose graph optimization (Bundle Adjustment)
• Triangulation of 3D points
• Output:
  • Reconstructed 3D points in .xyz format
  • Camera poses as 4×4 matrices
• Visualization using Open3D (view.py)

---

Installation

Dependencies

You must have the following installed:

• C++17 compatible compiler
• OpenCV (version ≥ 4.5)
• Ceres Solver (for Bundle Adjustment)
• CMake
• Python 3.10 (for visualization)
• Open3D

Depending on your platform, you may need to install additional libraries. Windows/macOS executables will be provided soon.

Python

You can use either Anaconda (or Miniconda) to create a virtual Python environment. The build scripts run pip install commands to set up the environment. All Python dependencies are listed in requirements.txt.

Installation Steps

1. Clone the repository:

   git clone https://github.com/Yarroudh/MinimalSfM
   cd MinimalSfM

2. Install prerequisites as mentioned above.

3. Run the build script to compile the project.

   ./build.sh  # Linux/macOS
   build.bat   # Windows

This will:

• Create a build/ directory
• Run CMake
• Compile the C++ code
• Generate the executable slam
• Install Python dependencies

---

Running the Program

./slam <image_directory> [output_points.xyz] [output_poses.txt]

• image_directory (required): Path to folder containing .jpg images.
• output_points.xyz (optional): Output 3D points (default: triangulated_points.xyz).
• output_poses.txt (optional): Output camera poses (default: camera_poses.txt).

Example

./slam ./data/sample points.xyz poses.txt

Data sample

You can use the sample images provided in the data/sample/ directory. The images are taken from the Caterpillar dataset, which is a common benchmark for SfM tasks.

---

Visualizing Results

Use the provided Python script view.py to view the 3D reconstruction:

python view.py [output_points.xyz] [output_poses.txt]

This script uses Open3D to render the point cloud and camera coordinate frames.

---

How It Works

1. Image Loading: Reads .jpg images from the given directory.
2. Feature Extraction: Detects SIFT keypoints and descriptors.
3. Matching: Performs FLANN-based nearest neighbor matching with ratio test.
4. Pose Estimation: Estimates relative motion using essential matrix.
5. Loop closure: Registers the first image to the last one.
6. Pose Graph Optimization: Runs Bundle Adjustment using Ceres Solver to optimize camera poses.
7. Triangulation: 3D points are triangulated using OpenCV.
8. Output: Saves reconstructed points and camera poses to disk.

---

Android Support

This project can be built for Android using the prebuilt Android SDKs of OpenCV and by cross-compiling Ceres Solver for Android. Visualization is not currently supported on Android, but the core SfM computation is portable. This is an ongoing area of development.

---

Contact

For questions or contributions, feel free to open an issue or submit a pull request.