CUDA GNC: run GncOptimizer with the CUDA SFM LM inner solver, ~20x over CPU GNC by leolrg · Pull Request #2593 · borglab/gtsam

leolrg · 2026-07-07T03:35:24Z

Summary

This PR makes GncOptimizer work with the CUDA SFM Levenberg-Marquardt backend as its
inner solver, adds a GNC benchmark mode to timeCudaSFMBAL, and extends the CUDA LM
profiling instrumentation used to measure it.

GncOptimizer<GncParams<CudaSfmLevenbergMarquardtParams>>(graph, values, params).optimize()
is the entire user story — GNC's template only requires an OptimizerType typedef and a
working optimize() from its base params type, which the CUDA optimizer already provided.

Changes

GNC enablement (2 small functional changes)

Accept zero-diagonal sqrt-information in CudaSfmProjectionBatch::IsUsableSqrtInfo.
GNC's makeWeightedGraph scales factor information by per-factor weights, and TLS
drives outlier weights to exactly 0; the conversion previously rejected the resulting
zero-information noise models. Zero rows are safe: kernels only multiply by sqrt-info,
and Schur point blocks stay invertible via lambda damping.
Add equals() to CudaSfmLevenbergMarquardtParams, completing the duck-typed
interface GncParams expects (CPU params inherit it from NonlinearOptimizerParams;
the CUDA params class is standalone).

Usage

Switching an existing GNC setup to the CUDA inner solver is a one-type change — swap
LevenbergMarquardtParams for CudaSfmLevenbergMarquardtParams:

#include <gtsam/nonlinear/GncOptimizer.h>
#include <gtsam/slam/cuda/CudaSfmLevenbergMarquardt.h>

using gtsam::cuda::CudaSfmLevenbergMarquardtParams;

// graph: NonlinearFactorGraph of GeneralSFMFactor<SfmCamera, Point3>
// (or BatchFactor<GeneralSFMFactor<SfmCamera, Point3>, 2>), possibly
// with a small number of corrupted measurements.
// initial: Values with SfmCamera / Point3 entries.

CudaSfmLevenbergMarquardtParams lmParams =
    CudaSfmLevenbergMarquardtParams::CeresDefaults();

GncParams<CudaSfmLevenbergMarquardtParams> gncParams(lmParams);
gncParams.setLossType(GncLossType::TLS);
// Optional: gncParams.setKnownInliers(...) e.g. for odometry-like factors.

GncOptimizer<GncParams<CudaSfmLevenbergMarquardtParams>> gnc(graph, initial, gncParams);
Values result = gnc.optimize();

// Per-factor weights: outliers -> 0 (TLS), inliers -> 1.
const Vector& weights = gnc.getWeights();
// Per-outer-iteration timing breakdown of the run:
const GncTiming& timing = gnc.getTiming();

Benchmarking

Per-outer-iteration stage timing in GncOptimizer (getTiming()): weight update,
weighted graph rebuild, base optimize, cost evaluation. Header-only, no behavior change.
--gnc cpu|cuda mode in timeCudaSFMBAL with deterministic outlier injection
(--gnc-outlier-fraction/-pixels/-seed, corrupting only tracks that keep >= 2 clean
views), --gnc-loss tls|gm, timing breakdown, and quality metrics against the injected
ground truth (classification precision/recall, inlier-only RMS reprojection error).
Extended CUDA LM timing artifacts: H2D/D2H transfer profiling (bytes, copy time,
host-build vs device-alloc split), per-iteration/per-attempt solve-loop profiles.

Benchmark results

Dubrovnik BAL datasets, 10% injected outliers at 12 px, TLS loss, seed 42, A100:

Dataset	Measurements	CPU GNC	CUDA GNC	Speedup	Precision / Recall (both)
dubrovnik-16	83,718	134.1 s	6.2 s	21.6x	0.943 / 0.964
dubrovnik-88	383,937	627.4 s	35.2 s	17.8x	0.857 / 0.988
dubrovnik-135	553,336	1078.6 s	54.0 s	20.0x	0.856 / 0.988

Testing

New tests in testCudaSfm: TLS and GM classification parity vs
GncOptimizer<GncParams<LevenbergMarquardtParams>> on a synthetic BAL problem with
injected outliers; zero-information weighted-graph regression test; params equals().
Full testCudaSfm (41 tests) and testGncOptimizer suites pass.

Reproduce benchmarks:
./timeCudaSFMBAL --gnc cuda [BALfile]
./timeCudaSFMBAL --gnc cpu [BALfile]

GncOptimizer<GncParams<CudaSfmLevenbergMarquardtParams>> now works as a drop-in: add the equals() method GncParams expects from its base params type, and accept zero-diagonal sqrt-information in the CUDA SFM conversion, since GNC weighted graphs scale factor information by weights that TLS drives to exactly zero. Add parity tests against CPU GNC (TLS and GM classification on a synthetic BAL problem with injected outliers) and a zero-information weighted-graph regression test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Instrument GncOptimizer::optimize() with per-outer-iteration stage timings (weight update, weighted graph rebuild, base optimize, cost evaluation), exposed via getTiming(). Add --gnc cpu|cuda to timeCudaSFMBAL with deterministic outlier injection (--gnc-outlier-fraction/-pixels/-seed, corrupting only tracks that keep >= 2 clean views), loss selection, per-iteration timing output, and outlier-classification precision/recall plus inlier-only RMS reprojection error against the injected ground truth. On dubrovnik 16/88/135 with 10% outliers at 12 px (TLS), CUDA GNC is 18-22x faster than CPU GNC at matched classification quality; >95% of remaining CUDA GNC time is host-side graph rebuild and per-iteration re-upload, motivating a device-resident GNC loop. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot

Pull request overview

Note

Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.

Enables running GncOptimizer with the CUDA SFM Levenberg–Marquardt backend and adds benchmarking/profiling instrumentation to compare CPU vs CUDA GNC performance.

Changes:

Adds GNC benchmarking mode to timeCudaSFMBAL (CPU/CUDA backends), including deterministic outlier injection and timing/quality metrics.
Extends CUDA LM profiling (transfer breakdown, per-iteration/per-attempt stats) and adds GNC per-outer-iteration timing.
Extends CUDA SFM graph conversion and utilities to support batched factors and fully down-weighted (zero-information) measurements.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
timing/timeSFMBAL.cpp	Adjusts batch SFM graph construction behavior.
timing/sfm_ba/timeCudaSFMBAL.cpp	Adds `--gnc` benchmarking mode, graph-kind options, and detailed reporting.
gtsam/slam/tests/testCudaSfm.cpp	Adds tests for batched conversion, timing artifacts, GNC parity, and zero-information regression.
gtsam/slam/cuda/cuda_sfm.i	Exposes additional CUDA LM profiling fields to bindings.
gtsam/slam/cuda/CudaSfmValues.h	Adds profiled pack/download paths with transfer breakdown.
gtsam/slam/cuda/CudaSfmProjectionBatch.h	Allows zero-information sqrt-info and adds transfer profiling.
gtsam/slam/cuda/CudaSfmLevenbergMarquardt.h	Adds params `equals()` and expands result profiling structs/fields.
gtsam/slam/cuda/CudaSfmLevenbergMarquardt.cu	Implements batched-factor conversion and detailed profiling instrumentation.
gtsam/nonlinear/cuda/DeviceValues.h	Adds optional upload timing and delta allocation timing.
gtsam/nonlinear/cuda/DeviceSparseNormalEquations.h	Adds optional upload profiling for sparsity pattern transfers.
gtsam/nonlinear/GncOptimizer.h	Adds per-iteration timing collection via `getTiming()`.
gtsam/nonlinear/BatchFactor.h	Exposes batch child factors for conversion/introspection.
gtsam/base/cuda/CudaDeviceArray.h	Adds profiled upload/download helpers and transfer summary structs.
.gitignore	Ignores benchmark log artifact directories.

+  std::mt19937 rng(seed);
+  std::shuffle(eligible.begin(), eligible.end(), rng);
+  std::uniform_real_distribution<double> angle(0.0, 2.0 * M_PI);


+  CudaDeviceTransferTiming uploadProfiled(
+      const std::vector<T>& host, cudaStream_t stream = nullptr) {
+    CudaDeviceTransferTiming timing;
+    timing.bytes = sizeof(T) * host.size();
+
+    const auto resizeStart = std::chrono::steady_clock::now();
+    resize(host.size());
+    timing.resizeElapsed = internal::CudaTransferElapsedSince(resizeStart);
+
+    if (host.empty()) return timing;
+    const auto copyStart = std::chrono::steady_clock::now();
+    GTSAM_CUDA_CHECK(cudaMemcpyAsync(data_, host.data(), sizeof(T) * size_,
+                                     cudaMemcpyHostToDevice, stream));
+    GTSAM_CUDA_CHECK(cudaStreamSynchronize(stream));
+    timing.copyElapsed = internal::CudaTransferElapsedSince(copyStart);
+    return timing;
+  }


      if (params.linearSolver == CudaSfmLinearSolverType::DenseSchur) {
+        stageStart = Clock::now();
        if (params.diagonalDamping) {
          denseSchurSolver.solve(current, batch, numCameras, lambda,
                                 dampingDiagonal, &delta, context.stream());
        } else {
          denseSchurSolver.solve(current, batch, numCameras, lambda, &delta,
                                 context.stream());
        }
+        attemptProfile.denseSchurSolveElapsed =
+            ElapsedSinceAfterSync(stageStart, context.stream());
+        result.denseSchurSolveElapsed +=
+            attemptProfile.denseSchurSolveElapsed;
      } else {


  Values optimize() {
+    using Clock = std::chrono::steady_clock;
+    const auto elapsedSince = [](Clock::time_point start) {
+      return std::chrono::duration<double>(Clock::now() - start).count();
+    };
+    timing_ = GncTiming();
+    const auto totalStart = Clock::now();


+    if (n <= 2) continue;
+    // Leave at least two clean measurements per track.
+    const size_t maxCorruptible = n - 2;
+    for (size_t m = 0; m < maxCorruptible; ++m) {
+      eligible.emplace_back(j, m);
+    }


ProfFan · 2026-07-08T18:09:21Z

+  double totalElapsed = 0.0;
+  std::vector<GncIterationTiming> iterations;
+
+  double sumWeightsUpdate() const {


What are these functions for?

dellaert · 2026-07-13T15:41:13Z

@leolrg Please evaluate Copilot review comments - and Fan's :-)

leolrg and others added 3 commits July 6, 2026 06:42

added more timing artifacts

293c122

dellaert requested a review from Copilot July 7, 2026 07:29

Copilot started reviewing on behalf of dellaert July 7, 2026 07:47 View session

Copilot AI reviewed Jul 7, 2026

View reviewed changes

ProfFan reviewed Jul 8, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

CUDA GNC: run GncOptimizer with the CUDA SFM LM inner solver, ~20x over CPU GNC#2593

CUDA GNC: run GncOptimizer with the CUDA SFM LM inner solver, ~20x over CPU GNC#2593
leolrg wants to merge 3 commits into
borglab:developfrom
leolrg:cuda-gnc

leolrg commented Jul 7, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

ProfFan Jul 8, 2026

Uh oh!

dellaert commented Jul 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

leolrg commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Usage

Benchmark results

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

ProfFan Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

dellaert commented Jul 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

leolrg commented Jul 7, 2026 •

edited

Loading