Skip to content

Training crash: "n_instances exceeds int range" (MCMC, 10M splats) #1306

Description

@msultan32

Description

Headless training crashes when MCMC densification reaches --max-cap with a large dataset.

Environment

  • LichtFeld Studio: v0.5.2-128-gc4eb3dea (tag c4eb3dea)
  • Platform: GCP Cloud Run, NVIDIA GPU, CUDA driver 13.0
  • Mode: --headless --train
  • Dataset loader: COLMAP (binary), 6 cameras, 816 images, 121,219 initial points

Command

LichtFeld-Studio \
  -d /path/to/lichtfeld_input \
  -o /path/to/training_output \
  -i 30000 \
  --headless --train \
  --mask-mode ignore \
  --enable-mip \
  --max-cap 10000000 \
  --strategy mcmc

Error

These are some of the logs from my pipeline:

Training [...] 16400/30000 | Loss: 0.0828 | Splats: 10000000 (+)
[PipelinedImageLoader] Shutting down...
[PipelinedImageLoader] Done: 16508 loaded, 15692 hits, 816 misses
[error] application.cpp:190 Training error: n_instances exceeds int range

Additional context

Same dataset trains successfully with lower --max-cap (e.g. 1M–2M).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtrainingGaussian splatting training

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions