feat: TRELLIS.2 image-to-3D backend — native game-ready pipeline (no nvdiffrast)

.github/workflows/deploy.yml

@@ -56,6 +56,8 @@ jobs:
         run: ./scripts/sync-doc-versions-from-cmake.sh --check
       - name: File association packaging files present
         run: chmod +x ./scripts/verify-file-associations.sh && ./scripts/verify-file-associations.sh
+      - name: TRELLIS.2 restricted-dependency gate (no nvdiffrast/nvdiffrec)
+        run: chmod +x ./scripts/check-trellis2-restricted-deps.sh && ./scripts/check-trellis2-restricted-deps.sh
 
 ####################################################################
 # Asset Scan (runs first, before all builds)

.gitignore

@@ -161,3 +161,5 @@ __pycache__/
 !docs/MOCAP_SPIKE.md
 .mocap_work/
 .venv-mocap/
+!docs/TRELLIS2.md
+!docs/trellis2-dependencies.md

CLAUDE.md

@@ -155,6 +155,9 @@ qtmesh generate3d image.png --texture-size 2048 -o out.glb  # quality pass (ON b
 qtmesh generate3d image.png --no-smooth --no-refine --no-bake-texture -o out.glb  # raw marching-cubes output with per-vertex color (pre-quality-pass behavior)
 qtmesh generate3d image.png --upscale-texture -o out.glb  # + Real-ESRGAN 2x on the baked diffuse (sharper color; upscale model downloads on demand)
 qtmesh generate3d image.png --no-pbr -o out.glb  # skip the PBR stage (#404 normal+roughness synthesized from the baked diffuse and bound into the material — ON by default; the polished-surface look; writes *_normal/_roughness.png sidecars)
+qtmesh generate3d photo.png -o out.glb --backend trellis2 --preset balanced --seed 42  # TRELLIS.2 backend (Microsoft, MIT code+weights): the highest-quality tier and the DEFAULT whenever its sidecar runtime is installed (ai/trellis2/install.py; Linux + NVIDIA GPU ≥24GB VRAM). Python does INFERENCE ONLY; QtMeshEditor natively does alpha matte (own U²-Net — upstream's RMBG-2.0 is CC-BY-NC and never loads), game-ready weld/debris-cull/simplify, xatlas UV + multi-channel PBR bake (basecolor RGBA/roughness/metallic/normal via Trellis2Bake — deliberately WITHOUT NVIDIA nvdiffrast/nvdiffrec, which are research-only-licensed and CI-gated out; audit: docs/trellis2-dependencies.md). Full-res source preserved as <out>_source.qtm3d
+qtmesh generate3d photo.png -o out.glb --backend trellis2 --preset high --target-tris 25000 --texture-size 4096  # TRELLIS.2 game-ready presets: --target-tris 10000/25000/50000 (0 = original density); presets fast=512 / balanced=1024_cascade / high=1536_cascade; QTMESH_TRELLIS2_MOCK=1 exercises the whole pipeline without a GPU
+qtmesh generate3d photo.png --target-tris 25000 -o out.glb  # game-ready pass, ALL backends: weld + debris-cull + meshopt-simplify toward the target, then re-bake the diffuse on the simplified mesh (TripoSR field bake is density-independent) + bake the dense source's relief into a tangent-space detail normal map sharing the same atlas (Trellis2Bake::bakeDetailNormal). This is the fix for 'decimated Tripo output turns into a blob / skins badly' — simplify hard, keep detail in textures. 0 = original density
 qtmesh generate3d image.png --backend triposg --flow-steps 25 --guidance 7 -o out.glb  # TripoSG backend (1.5B rectified-flow DiT, MIT): higher-fidelity GEOMETRY, slower; models download on first use; --guidance 0 disables CFG. TripoSG is GEOMETRY-ONLY (no colour decoder) — a colour bake queries TripoSR's image-conditioned colour field on the same image + projects the input photo onto the visible front (the back is inferred, so it's approximate; a "Generate texture (AI)" option in the GUI does a front-photo + SD-generated-back multi-view bake for a better back). int8 tier is DROPPED for TripoSG (fp32 only — quantized geometry degrades to blobs, no ARM speed win)
 qtmesh segment model.fbx                       # AI part segmentation (#410/#818): per-part vertex/face counts; category auto-detected (body/vegetation/vehicle/building)
 qtmesh segment model.fbx --json                # full vertex/face → label arrays + per-part summary + resolved category (stable schema)

THIRD_PARTY_AI_MODELS.md

@@ -127,6 +127,49 @@ the binary). Attribution + licenses for the models and their training data:
   `scripts/upload-triposr-models.sh`). First use downloads them; if ever absent the
   feature reports a clean "not yet hosted" state (no crash) — the RigNet precedent.
 
+## TRELLIS.2 — image-to-3D generation, sidecar backend (default when installed)
+
+- **Model:** Microsoft TRELLIS.2 — sparse-voxel "O-Voxel" flexible-dual-grid
+  representation with volumetric PBR attributes (base color / metallic /
+  roughness / alpha), 4B-parameter flow stack. NOT an ONNX consumer: it runs as
+  an out-of-process **Python sidecar** (`ai/trellis2/`, Linux + NVIDIA CUDA,
+  ≥24 GB VRAM recommended) because the custom sparse CUDA kernels
+  (FlexGEMM/o-voxel/CuMesh) have no ONNX lowering. First runtime-Python
+  component in the project; nothing is bundled — the user installs the isolated
+  environment with `ai/trellis2/install.py`.
+- **Source:** Microsoft — *"Native and Compact Structured Latents for 3D
+  Generation"* (arXiv 2512.14692). https://github.com/microsoft/TRELLIS.2 —
+  code **MIT**, pinned `75fbf0183001ed9876c8dbb35de6b68552ee08bd`. Weights:
+  https://huggingface.co/microsoft/TRELLIS.2-4B — **MIT**, rev `af44b45f…`,
+  ≈18.9 GB, downloaded on first generation under the user's HF account (plus
+  the MIT `microsoft/TRELLIS-image-large` sparse-structure decoder).
+  Companion libs JeffreyXiang/CuMesh + FlexGEMM — **MIT**, pinned in install.py.
+- **License boundary (the deciding work of this integration):** upstream's
+  texture bake + preview renderers use NVIDIA **nvdiffrast/nvdiffrec** (NVIDIA
+  Source Code License — research/evaluation only) — both are **excluded
+  entirely** (not installed/imported/invoked; `install.py` patches the MIT file
+  `o_voxel/__init__.py` so the package imports without them; CI gate
+  `scripts/check-trellis2-restricted-deps.sh` + `Trellis2GuardTest`). Their
+  functionality is QtMeshEditor-native code: `src/ImageTo3D/Trellis2Bake.{h,cpp}`
+  (xatlas unwrap + UV-space barycentric rasterizer + Ericson closest-point +
+  trilinear sparse-volume sampling + meshoptimizer game-ready pipeline).
+  The upstream default background remover `briaai/RMBG-2.0` is **CC BY-NC** and
+  is never downloaded — the loader is stubbed and the alpha matte comes from
+  the project's own U²-Net (`BackgroundRemover` keepAlpha). The conditioning
+  encoder is **DINOv3** (`facebook/dinov3-vitl16-pretrain-lvd1689m`) under
+  Meta's custom **DINOv3 License** — commercial use permitted, gated download,
+  "Built with DINOv3" attribution; NOT MIT, so the stack must never be
+  described as "entirely MIT". Full audit table + pins:
+  `docs/trellis2-dependencies.md`; user guide: `docs/TRELLIS2.md`.
+- Sidecar contract: `generate.py` emits raw vertices/faces + the sparse
+  attribute volume as a **QTM3D interchange** file
+  (`src/ImageTo3D/Trellis2Interchange.{h,cpp}`); `Trellis2Predictor` drives the
+  process (JSON-line progress, cancellation, runtime discovery via
+  `QTMESH_TRELLIS2_ENV`/`ai/trellis2Env`, mock mode for GPU-less e2e tests) and
+  is the **default backend** (`MeshGenPredictor::defaultBackend()`) whenever
+  the runtime resolves; every surface reports a clean "runtime not installed"
+  message otherwise (no crash).
+
 ## U²-Net — background removal for image-to-3D (epic #764)
 
 - **Model:** U²-Net salient-object detection (`u2net.onnx`) — the default

ai/trellis2/README.md

@@ -0,0 +1,79 @@
+# QtMeshEditor TRELLIS.2 sidecar
+
+Out-of-process Python runtime for the `trellis2` image-to-3D backend
+(`qtmesh generate3d --backend trellis2`, the "TRELLIS.2 — High Quality" option in the
+GUI's *AI: Image → 3D* panel).
+
+## What it does — and what it deliberately does NOT do
+
+`generate.py` runs **Microsoft TRELLIS.2** (MIT, pinned revision) inference only:
+
+```text
+RGBA image (alpha matte made by QtMeshEditor's own U²-Net)
+   ↓ generate.py            DINOv3 cond → sparse structure → shape SLat → tex SLat
+raw geometry + sparse PBR attribute volume (base color / metallic / roughness / alpha)
+   ↓ QTM3D interchange file (qtm3d.py ↔ src/ImageTo3D/Trellis2Interchange.*)
+QtMeshEditor C++: cleanup → weld → simplify (game-ready presets) → xatlas UV
+   → texture/PBR bake (own rasterizer) → normals/tangents → Ogre / GLB / FBX
+```
+
+The upstream reference implementation uses **NVIDIA nvdiffrast + nvdiffrec** for UV-space
+rasterization, texture baking and PBR previews. Those libraries are under the NVIDIA Source
+Code License (research/evaluation only) and are **prohibited here**: they are not in
+`requirements.txt`, `install.py` never installs them, `generate.py` never imports them (and
+warns — `--strict`: fails — if they are unexpectedly present), and `install.py` patches the
+MIT file `o_voxel/__init__.py` so the o-voxel package imports without them. The equivalent
+functionality is QtMeshEditor's own C++ code (`src/ImageTo3D/Trellis2Bake.*`). Full audit:
+`docs/trellis2-dependencies.md`.
+
+The upstream default background remover (`briaai/RMBG-2.0`, named in the shipped
+`pipeline.json`) is **CC BY-NC** and is likewise never downloaded or loaded — the loader is
+stubbed and inputs must already carry an alpha matte.
+
+## Requirements
+
+- Linux, NVIDIA GPU (**≥ 24 GB VRAM** recommended; `low_vram` staggering is on), CUDA 12.4
+- Python ≥ 3.10, git, a compiler toolchain (the o-voxel/CuMesh/FlexGEMM CUDA extensions are
+  built from pinned sources)
+- A Hugging Face account that has **accepted Meta's DINOv3 License** (the
+  `facebook/dinov3-vitl16-pretrain-lvd1689m` encoder is gated): `huggingface-cli login`
+- ~19 GB for TRELLIS.2-4B weights (MIT), downloaded on first generation
+
+## Install
+
+```bash
+python3 ai/trellis2/install.py                 # → <app data>/QtMeshEditor/trellis2
+# or choose a location:
+python3 ai/trellis2/install.py --dest /opt/qtmesh-trellis2
+```
+
+QtMeshEditor auto-detects the default location. For a custom one, set the env var
+`QTMESH_TRELLIS2_ENV=<dest>` or QSettings `ai/trellis2Env`.
+
+## Direct use / troubleshooting
+
+```bash
+ENV=~/.local/share/QtMeshEditor/trellis2
+$ENV/env/bin/python $ENV/generate.py --check                 # environment probe
+$ENV/env/bin/python $ENV/generate.py --report-deps --check   # dependency report
+$ENV/env/bin/python $ENV/generate.py --input subject.png --output out.qtm3d \
+    --preset balanced --seed 42
+$ENV/env/bin/python $ENV/generate.py --input any.png --output out.qtm3d --mock
+                                                             # plumbing test, no GPU
+```
+
+Presets: `fast` = TRELLIS.2 `512`, `balanced` = `1024_cascade` (upstream default),
+`high` = `1536_cascade`.
+
+Progress/status is emitted as JSON lines on stdout (QtMeshEditor parses these); tqdm and
+debug logs go to stderr.
+
+## Licenses
+
+See `THIRD_PARTY_LICENSES.md` (this directory) and `docs/trellis2-dependencies.md` for the
+authoritative table. Summary: TRELLIS.2 code + weights MIT; CuMesh/FlexGEMM MIT; PyTorch /
+flash-attn BSD-3; transformers Apache-2.0; DINOv3 encoder under Meta's **DINOv3 License**
+(commercial use permitted, gated download, "Built with DINOv3" attribution); easydict
+LGPL-3.0 (pure-Python, flagged); nvdiffrast/nvdiffrec/RMBG-2.0 excluded.
+
+*Built with DINOv3.*

ai/trellis2/THIRD_PARTY_LICENSES.md

@@ -0,0 +1,30 @@
+# TRELLIS.2 sidecar — third-party licenses
+
+This environment is installed on the user's machine by `install.py`; QtMeshEditor does not
+redistribute any of it. Authoritative audit: `docs/trellis2-dependencies.md` (pinned
+revisions, red/yellow flags, enforcement).
+
+| Component | License | Notes |
+|---|---|---|
+| Microsoft TRELLIS.2 (code, incl. in-repo `o-voxel`; pinned `75fbf018…`) | MIT | `o_voxel/__init__.py` locally patched (lazy `postprocess`) so the package imports without nvdiffrast |
+| `microsoft/TRELLIS.2-4B` weights (HF rev `af44b45f…`) | MIT | downloaded on first use |
+| `microsoft/TRELLIS-image-large` sparse-structure decoder (HF rev `25e0d31f…`) | MIT | referenced by upstream `pipeline.json` |
+| JeffreyXiang/CuMesh (pinned `12289e10…`) | MIT | built from source — never `pip install cumesh` (unrelated unlicensed PyPI package) |
+| JeffreyXiang/FlexGEMM (pinned `6dd94a85…`) | MIT | sparse conv + `grid_sample_3d` |
+| Eigen (vendored inside o-voxel) | MPL-2.0 | |
+| PyTorch 2.6.0 / torchvision 0.21.0 | BSD-3-Clause | CUDA build |
+| flash-attn 2.7.3 (or xformers) | BSD-3-Clause | attention backend |
+| transformers, huggingface_hub, safetensors | Apache-2.0 | |
+| numpy | BSD-3-Clause | |
+| Pillow | HPND/MIT-CMU | |
+| easydict 1.13 | **LGPL-3.0** | pure-Python, imported unmodified from this user-installed env; the only copyleft item — flagged in the audit |
+| ninja, packaging | Apache-2.0 / BSD | build-time |
+| `facebook/dinov3-vitl16-pretrain-lvd1689m` | **DINOv3 License** (Meta, custom) | commercial use permitted; gated download under the user's HF account; license text: <https://ai.meta.com/resources/models-and-libraries/dinov3-license/>. **Built with DINOv3.** |
+
+## Intentionally excluded
+
+| Component | License | Why excluded |
+|---|---|---|
+| nvdiffrast | NVIDIA Source Code License (1-Way Commercial) — research/evaluation only | replaced by QtMeshEditor's own C++ rasterizer/baker (`src/ImageTo3D/Trellis2Bake.*`) |
+| nvdiffrec (`nvdiffrec_render`) | NVIDIA Source Code License for nvdiffrec — research/evaluation only | preview lighting replaced by QtMeshEditor's Ogre/RTSS + HDR/IBL renderer |
+| `briaai/RMBG-2.0` background remover | CC BY-NC 4.0 | never downloaded/loaded; background removal is done by QtMeshEditor's U²-Net (Apache-2.0) |