Skip to content

MeshTransform: Slice E — Spike: MHR + SAM 3D Body (photo→rigged human) licenses, ONNX export, momentum eval #850

Description

@fernandotonon

Part of #845 · Design: docs/MESH_TRANSFORM_DESIGN.md §3.2 · Timebox: 1 week · Gate for Slice F.

Feasibility spike for photo → rigged parametric human on the permissive Meta stack (the SMPL replacement):

  • MHR — Momentum Human Rig (facebookresearch/MHR, Nov 2025): parametric full-body model, artist topology, RIGGED + SKINNED, 7 LODs. Survey read: Apache-2.0 code AND model, single unified license.
  • SAM 3D Body (facebookresearch/sam-3d-body): feed-forward single-photo → MHR shape/pose params. "SAM License", commercial-permitted per the survey — exact redistribution terms unverified.
  • momentum — the C++ evaluation lib MHR rides on.

Deliverables (memo filed as a comment on #845, the #818-C5 pattern):

  • License verbatim quotes on the EXACT weight/asset files (MHR model asset incl. topology/rig data, SAM 3D Body checkpoint, momentum lib) → draft THIRD_PARTY_AI_MODELS.md entries
  • SAM 3D Body ONNX export attempt: transformer estimator (DINO-style backbone + param head) — expect exportable; risks are detection pre/post-processing and any iterative refinement loop. Deliver exported graph(s) + parity numbers, or the documented blocker
  • MHR C++ evaluation path: momentum as a dep (weight it) vs porting param→mesh math (blendshape matrix + LBS — primitives we have). Asset size across the 7 LODs
  • Skeleton audit: map MHR joints → the 22-joint canonical (MotionInbetween::canonicalIndexForBone) so generated humans play the motion library
  • Go/no-go + effort estimate for Slice F

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai-assistLocal-AI-assisted 3D workflows (epic prefix: AI:)enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions