Skip to content

Commit 62bed60

Browse files
committed
Move MrIMU plugin to plugins/ directory and update start.sh
- Moved MrIMU plugin from root to plugins/MrIMU/ following Meshroom plugin structure - Updated start.sh to point MESHROOM_PLUGINS_PATH to new location - Plugin structure now matches Meshroom documentation requirements - All plugin files verified and imports working correctly
1 parent 9300a33 commit 62bed60

File tree

11 files changed

+1428
-5
lines changed

11 files changed

+1428
-5
lines changed

plugins/MrIMU/README.md

Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
# MrIMU - IMU Integration Plugin for Meshroom
2+
3+
MrIMU is a Meshroom plugin that integrates IMU (Inertial Measurement Unit) data from accelerometers and gyroscopes into photogrammetry workflows. This plugin helps improve camera pose estimation by constraining orientations using IMU measurements, particularly useful for maintaining vertical axis stability.
4+
5+
## Features
6+
7+
- **Load IMU Data**: Support for OpenCamera-Sensors CSV files and CAMM metadata from GoPro MP4 files
8+
- **Gravity Vector Extraction**: Uses low-pass Butterworth filter to extract gravity direction from accelerometer data
9+
- **Camera Pose Correction**: Applies IMU orientation constraints to StructureFromMotion camera poses
10+
- **Adjustable IMU Influence**: Balance between optical and IMU data with configurable weight (0.0 to 1.0)
11+
- **Z-Axis Locking**: Option to keep vertical axis aligned with gravity for stable reconstructions
12+
13+
## Plugin Structure
14+
15+
The MrIMU plugin follows Meshroom's plugin structure requirements:
16+
17+
```
18+
MrIMU/
19+
├── meshroom/ # Meshroom nodes and pipelines
20+
│ ├── __init__.py # Plugin module initialization
21+
│ ├── config.json # Optional plugin configuration
22+
│ └── nodes/ # Node definitions
23+
│ ├── __init__.py # Required to be a Python module
24+
│ ├── LoadIMUData.py # Load IMU data node
25+
│ ├── ApplyIMUConstraints.py # Apply IMU constraints node
26+
│ └── imu_utils.py # IMU processing utilities
27+
├── requirements.txt # Python dependencies
28+
└── README.md # This file
29+
```
30+
31+
## Installation
32+
33+
### Prerequisites
34+
35+
- Meshroom installed and configured
36+
- Python packages: `numpy`, `scipy`
37+
38+
### Installation Steps
39+
40+
1. **Clone or download this plugin** to a directory of your choice:
41+
```bash
42+
cd /path/to/plugins
43+
git clone <repository-url> MrIMU
44+
# or download and extract the plugin
45+
```
46+
47+
2. **Set the MESHROOM_PLUGINS_PATH environment variable**:
48+
49+
On Linux:
50+
```bash
51+
export MESHROOM_PLUGINS_PATH=/path/to/plugins/MrIMU:$MESHROOM_PLUGINS_PATH
52+
```
53+
54+
On Windows:
55+
```cmd
56+
set MESHROOM_PLUGINS_PATH=C:\path\to\plugins\MrIMU;%MESHROOM_PLUGINS_PATH%
57+
```
58+
59+
3. **Install Python dependencies**:
60+
61+
Option A: Install globally (if not already installed):
62+
```bash
63+
pip install numpy scipy
64+
```
65+
66+
Option B: Create a virtual environment in the plugin directory:
67+
```bash
68+
cd MrIMU
69+
python3 -m venv venv
70+
source venv/bin/activate # On Windows: venv\Scripts\activate
71+
pip install -r requirements.txt
72+
```
73+
Meshroom will automatically use the `venv/` directory if it exists.
74+
75+
4. **Restart Meshroom** to load the plugin
76+
77+
5. **Verify installation**: The plugin nodes should appear in the node library under the "IMU" category:
78+
- `LoadIMUData`
79+
- `ApplyIMUConstraints`
80+
81+
## Usage
82+
83+
### Workflow Overview
84+
85+
1. **Load IMU Data**: Use `LoadIMUData` node to process IMU CSV files or extract CAMM from MP4
86+
2. **Run StructureFromMotion**: Execute Meshroom's standard SfM pipeline
87+
3. **Apply IMU Constraints**: Use `ApplyIMUConstraints` node to correct camera poses with IMU data
88+
89+
### Node: LoadIMUData
90+
91+
Loads and processes IMU data from CSV files or MP4 videos.
92+
93+
#### Input Parameters
94+
95+
- **Video File** (optional): MP4 video file for CAMM extraction (GoPro format)
96+
- **IMU Base Path**: Base path for OpenCamera-Sensors CSV files (without extension)
97+
- Expected files: `{basename}_accel.csv`, `{basename}_gyro.csv`, `{basename}_timestamps.csv`
98+
- Each CSV should have columns: `X`, `Y`, `Z`, `timestamp_ns`
99+
- **IMU Format**: Choose between `opencamera` (CSV) or `camm` (MP4)
100+
- **Gravity Filter Cutoff (Hz)**: Cutoff frequency for low-pass Butterworth filter (default: 0.1 Hz)
101+
- **IMU Sampling Rate (Hz)**: Sampling rate of IMU data (default: 100.0 Hz)
102+
103+
#### Output Files
104+
105+
- **IMU Data**: Processed IMU data in JSON format
106+
- **Gravity Vector**: Extracted gravity vector in NPY format (normalized, in world frame)
107+
108+
#### Example: OpenCamera-Sensors CSV Format
109+
110+
If your base path is `/data/my_session`, the plugin expects:
111+
- `/data/my_session_accel.csv`
112+
- `/data/my_session_gyro.csv`
113+
- `/data/my_session_timestamps.csv`
114+
115+
Each CSV file should have the following structure:
116+
```csv
117+
X,Y,Z,timestamp_ns
118+
0.123,0.456,9.789,1234567890000000000
119+
0.124,0.457,9.790,1234567890100000000
120+
...
121+
```
122+
123+
### Node: ApplyIMUConstraints
124+
125+
Applies IMU orientation constraints to StructureFromMotion camera poses.
126+
127+
#### Input Parameters
128+
129+
- **SfM Scene**: Input SfM scene file from StructureFromMotion node
130+
- **IMU Data**: Processed IMU data JSON file from LoadIMUData node
131+
- **IMU Weight**: Weight for IMU data influence (0.0 = optical only, 1.0 = IMU only, default: 0.5)
132+
- **Lock Z-Axis to Gravity**: Constrain vertical axis to align with gravity direction (default: True)
133+
134+
#### Output Files
135+
136+
- **Output Scene**: Corrected SfM scene file with IMU-adjusted camera poses
137+
138+
#### Usage Tips
139+
140+
- **Low IMU Weight (0.0-0.3)**: Use when optical data is reliable, IMU provides gentle guidance
141+
- **Medium IMU Weight (0.4-0.6)**: Balanced approach, good for most scenarios
142+
- **High IMU Weight (0.7-1.0)**: Use when optical tracking is poor but IMU data is reliable
143+
- **Lock Z-Axis**: Enable for scenes where vertical stability is critical (architecture, indoor scans)
144+
145+
## Coordinate Systems
146+
147+
The plugin handles coordinate transformations between:
148+
149+
- **Android Sensor Frame** (OpenCamera-Sensors):
150+
- X: points east
151+
- Y: points north
152+
- Z: points up
153+
154+
- **World Frame** (Meshroom/AliceVision):
155+
- X: points right
156+
- Y: points down
157+
- Z: points forward
158+
159+
Transformations are automatically applied when processing IMU data.
160+
161+
## Technical Details
162+
163+
### Gravity Vector Extraction
164+
165+
The gravity vector is extracted using a 4th-order Butterworth low-pass filter applied to accelerometer data. This filters out high-frequency motion while preserving the constant gravity component.
166+
167+
### IMU Constraint Application
168+
169+
Camera poses are corrected by:
170+
1. Extracting IMU-derived orientation from gravity and gyroscope data
171+
2. Blending IMU orientation with optical SfM orientation based on IMU weight
172+
3. Optionally locking Z-axis to gravity direction for vertical stability
173+
4. Orthonormalizing the resulting rotation matrix
174+
175+
## Troubleshooting
176+
177+
### Common Issues
178+
179+
1. **"Missing IMU CSV files" error**
180+
- Verify file names match expected pattern: `{basename}_accel.csv`, etc.
181+
- Check that CSV files contain required columns: `X`, `Y`, `Z`, `timestamp_ns`
182+
183+
2. **"Could not extract CAMM data" error**
184+
- CAMM extraction from MP4 files requires proper MP4 box parsing
185+
- Currently, full CAMM extraction is not implemented
186+
- Use OpenCamera-Sensors CSV format instead
187+
188+
3. **"No valid IMU data found" error**
189+
- Check that CSV files are not empty
190+
- Verify data format matches expected structure
191+
- Ensure timestamps are in nanoseconds
192+
193+
4. **Plugin not appearing in Meshroom**
194+
- Verify `MESHROOM_PLUGINS_PATH` is set correctly
195+
- Check that plugin directory contains `meshroom/` subdirectory
196+
- Restart Meshroom after setting environment variable
197+
- Check Meshroom logs for plugin loading errors
198+
199+
## Limitations
200+
201+
- CAMM extraction from MP4 files is not fully implemented (placeholder)
202+
- SfM scene format compatibility may vary with Meshroom versions
203+
- IMU data must be synchronized with image capture timestamps (manual synchronization may be required)
204+
205+
## Contributing
206+
207+
Contributions are welcome! Please ensure:
208+
- Code follows Meshroom plugin conventions
209+
- Error handling and logging are included
210+
- Documentation is updated
211+
212+
## License
213+
214+
This plugin follows the same license as Meshroom (MPL-2.0).
215+
216+
## References
217+
218+
- [Meshroom Plugin Documentation](https://github.com/alicevision/meshroom/wiki/Plugins)
219+
- [OpenCamera-Sensors](https://github.com/almalence/OpenCamera-Sensors)
220+
- [CAMM Metadata Specification](https://github.com/gopro/gpmf-parser)
221+

0 commit comments

Comments
 (0)