Skip to content

feat(sim): add deformable sock prop - #729

Draft
axelpey wants to merge 7 commits into
mainfrom
codex/sim-sock-softbody
Draft

feat(sim): add deformable sock prop#729
axelpey wants to merge 7 commits into
mainfrom
codex/sim-sock-softbody

Conversation

@axelpey

@axelpey axelpey commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a deformable MuJoCo surface-flex sock with rest-dihedral bending and simulator prop lifecycle support
  • render the original 2,496-vertex surface from a 109-vertex physics cage using smooth ISK2 skinning
  • package a simple two-tone gray sock through the simulator asset image
  • give the cloth realistic contact thickness and priority so the gripper does not pass as far through it

Validation

  • I ran the relevant local validation, or explained why it was skipped.
    • 68 launcher tests passed
    • 7 deformable viewer tests passed
    • Ruff passed on the Python changes
    • three-second headless drop remained finite
    • controlled pinch reduced maximum measured finger penetration from 2.37 mm to 0.92 mm and increased active finger contacts from 35 to 57
  • If I changed simulator behavior, config, or assets, I checked that the simulator starts with ./innate-sim up.
    • the branch was run interactively in the viewer and the sock was grasped; the final contact-only adjustment was also exercised headlessly against the real MARS gripper geometry
  • If I changed simulator asset files or asset references, I published the asset pack and committed the updated sim/assets.lock.json.
    • this asset path is pinned in ci/seed_asset_context.py; Publish Simulator Images and Publish Viewer Bundle are green, and the geometry image is available in GHCR

Handoff: please try this grasp

git fetch origin
git switch codex/sim-sock-softbody
./innate-sim up

Place Soft sock, pinch it around the heel/body with both gripper fingers, then move the arm while holding it. Please check:

  • whether either finger visibly passes through the rendered sock
  • whether the sock stays retained while the arm moves
  • whether folding remains stable around the cuff and heel
  • whether the slowdown during sustained contact feels acceptable

Measured locally, free/drop simulation had about 2.1x real-time capacity. A heavy active pinch ran around 0.66x real time. Denser physics cages were tested and rejected because they reduced performance as far as 0.22x real time.

@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds a deformable MuJoCo sock prop and its simulator lifecycle, browser skinning and streaming pipeline, packaged assets, and delivery validation.

  • Adds surface-flex physics with custom rest-dihedral bending and contact handling.
  • Streams timestamped control vertices to the browser and skins the high-resolution render mesh.
  • Packages pinned sock assets and extends asset-image verification and tests.

Confidence Score: 4/5

The PR is not yet safe to merge because the sock remains temporally ahead of the gripper during delayed playback.

The rigid robot and prop state is rendered on a delayed simulation timeline, but deformable updates still retain and apply only the newest frame, so motion while grasping the sock can produce visible sliding or finger penetration.

Files Needing Attention: sim/viewer/src/simSession.ts

Important Files Changed

Filename Overview
ros2_ws/src/mars_bot/mars_sim_driver/mars_sim_driver/softbody.py Introduces the deformable prop model, binding lifecycle, contact controls, and rest-dihedral force calculation.
ros2_ws/src/mars_bot/mars_sim_driver/mars_sim_driver/world_server.py Adds negotiated binary streaming of timestamped deformable control-mesh frames.
sim/viewer/src/simSession.ts Connects deformable frame reception to scene updates while coordinating the existing rigid-state playback path.
sim/viewer/src/deformableSkin.ts Adds browser-side skinning from the low-resolution physics cage to the detailed sock mesh.
sim/asset_tools/build_soft_sock_asset.py Builds the physics cage, bending data, render mesh, and compact skinning map from the authored sock source.
ci/verify_assets_image.py Extends published asset-layer validation to require the sock physics and viewer artifacts.

Reviews (3): Last reviewed commit: "fix(sim): localize soft sock deformation..." | Re-trigger Greptile

Comment on lines +464 to +468
for (const id of this.#deformablesDirty) {
const frame = this.#deformables.get(id);
if (frame) scene.setDeformableFrame(frame);
}
this.#deformablesDirty.clear();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Deformable playback bypasses timeline

When the robot moves while grasping the sock, these lines apply the latest deformable frame after rendering the robot on the delayed playback timeline. This places the sock 25–250 ms ahead of the gripper, causing visible sliding or finger penetration during motion; buffer and select deformable frames by their included simulation timestamp before applying them.

Knowledge Base Used: Simulation viewer and world state

@axelpey
axelpey force-pushed the codex/sim-sock-softbody branch from d8ec95e to 307a8c0 Compare August 31, 2026 09:00
@axelpey
axelpey force-pushed the codex/sim-sock-softbody branch from 307a8c0 to ead1468 Compare August 31, 2026 09:32
@axelpey
axelpey marked this pull request as draft September 3, 2026 22:25
@axelpey axelpey added the 0.7.5 label Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant