Proposal: optional experimental focus-aware stereo depth helper

[DNYoussef]

Hi @edin45 — before I write any code, I’d like to ask whether this would fit the project’s scope.

Use case

I’m investigating an offline depth workflow for explicitly paired captures from heterogeneous phone cameras, such as main and tele cameras.

The defensible first stage is calibrated stereo. A later experimental stage would test whether focus/blur residuals improve stereo depth or confidence after correspondence has already been established. It would not claim to replace stereo matching or reproduce a same-aperture biological sensor.

Proposed boundary

I would keep the scientific backend as a standalone command-line helper with:

• an explicit capture-bundle and calibration format
• CPU-only stereo as the baseline
• JSON Lines progress, warnings, metrics, and completion events
• cancellation and meaningful exit codes
• standard depth, confidence, PLY, metrics, and run-manifest outputs
• no models, hidden downloads, or CUDA requirement

The Flutter project would eventually receive only a thin optional adapter. Existing photogrammetry and splatting workflows would remain unchanged, and the feature would be labelled Experimental Focus-Aware Stereo Depth.

I would first develop and validate the helper in a companion repository. I would only propose the GUI adapter after the stereo workflow is independently useful and held-out tests show whether the focus-aware term provides a practical improvement. A negative focus result would leave a conventional paired-camera stereo helper rather than burdening this project with an unsupported feature.

Proposed stages

1. Capture schema, validator, tiny fixture, mock JSONL backend
2. CPU heterogeneous-stereo baseline and tests
3. Synthetic differential-defocus validation and required controls
4. Held-out real-device benchmarks
5. Small Flutter adapter, if the earlier gates pass
6. Packaging only as a separate decision

Mobile capture apps, real-time depth, neural monocular depth, and changes to COLMAP/OpenMVS are outside this proposal.

Maintenance

I would expect to own the experimental backend and its scientific tests. The GUI adapter would use a small, versioned process protocol so maintaining the Flutter side would not require understanding the research implementation.

Questions

1. Is an optional paired-camera bundle-to-depth workflow in scope for this GUI?
2. Would you prefer the backend to stay in a companion repository, with only an adapter proposed here after validation?
3. Is a user-installed external helper acceptable, or would you eventually require Windows/Nix bundling?
4. Should an adapter wait until the process/progress direction in feat: import clear photogrammetry frames from video #13 is resolved?
5. Which platforms would an eventual adapter need to support?
6. Would standard depth maps and PLY output be useful even without direct ingestion into the existing OpenMVS/Brush pipeline?

If this is outside the project’s intended scope, no problem—I can keep the work entirely separate.

[edin45]

Hey there!

That sounds really interesting, however I do feel like it does go against the core idea of the program, which is making the whole process simple, and straightforward to use.

The idea sounds super cool - but I feel like the user experience is going to be a little difficult - like getting the frames from both cameras + I also worry that with the variation between phone models, only some phone models will work.. etc...

But what do you think? Are these worries warranted?