Skip to content

Say what was substituted, and drive the joints the assets specify - #46

Merged
ZheningHuang merged 2 commits into
mainfrom
fix/simready-honest-reporting
Sep 9, 2026
Merged

Say what was substituted, and drive the joints the assets specify#46
ZheningHuang merged 2 commits into
mainfrom
fix/simready-honest-reporting

Conversation

@ZheningHuang

Copy link
Copy Markdown
Member

Stacked on #45. Found running that branch end to end on Office-Elliott from a raw scan — ingest through to a MuJoCo scene, nothing reused.

#45 holds up. The seed export gave 99/99 colliders from compiled sidecars — no fallbacks, no rejections — and both seed and authored scenes pass the stability harness: 0 contacts and 0 mm penetration at t=0, no BADQACC/CONTACTFULL, nothing escaped over 30 s × 8 randomised resets. The articulation is real and authored: a door leaf on a 0–90° hinge, a sit-stand desk on a 0–0.28 m prismatic lift, three window sashes. The sim/ collider-overwrite data-loss bug is genuinely fixed — the two chair clusters' collider files are name-prefixed and hash differently.

What this PR fixes is what happened around that.

The pipeline reported success it had not earned

I hit this three times in one run.

Reference generation. My first run produced error_fallback for every object — "OpenAI image generation needs $OPENAI_API_KEY" — and carried on building assets from raw evidence sheets. The only signal was objects: 2 (0 references ok), which reads as a count rather than an error.

The export under-reported what it invented. _object_physics returned None before recording anything when an object had no source_glb — which is every fixture the authoring agent writes into Room.py. So the authored room reported an empty no_sidecar while most of its physics had been derived at export, and the simulate stage's warning loop, keyed on exactly that field, never fired. Re-exporting the authored Office-Elliott room:

colliders: 229   sidecar_colliders: 99   derived_colliders: 130   sidecar_coverage: 0.432
authored_no_package: 18  [AirCon0, Detector0, Luminaire0, ...]

The stage now says, whether or not anything failed:

130 of 229 colliders (57%) were derived at export time, not read from a compiled sidecar — mass, friction and pivots for those came from a category table

A truncated authoring run looked finished. The step budget lands gracefully and exits 0 through the same path as a room the model considered done. It now reports the truncation, and --author-steps N exposes the budget.

Config that was documented but not wired

settings.as_environment() forwarded OPENAI_API_KEY and never GEMINI_API_KEY/LR_IMAGE_PROVIDER, so the README's Gemini path could not work from .env — I only got the run going by exporting both into the shell. sanity.py separately hard-failed on a missing OpenAI key whatever the provider, and would have passed an OpenAI key sitting alongside LR_IMAGE_PROVIDER=gemini before failing at the first object.

The compiled physics was not reaching the simulator

No actuators on any object joint. nu was 3, all room shake drives. The sidecars carry effort and velocity; grep effort in the exporter hit only a dataclass default. Now every sidecar joint gets a motor at its stated effort:

act_Door0_Door_Leaf               ctrlrange=[-45,45]     vel=2
act_Table1_Desk_Lift              ctrlrange=[-800,800]   vel=0.04
act_Window0_Sash_Casement_Left    ctrlrange=[-40,40]     vel=2

A motor, not a position servo, deliberately — a servo holds a setpoint, so it would clamp every door shut at ctrl=0 and a hung door would stop swinging under the shake. Passive drift is unchanged (1.3 mm vs 1.0 mm, and that difference is the heavier chairs); driven at full effort every joint reaches 100% of its range. Joints recovered from the raw extras get no actuator, because nothing stated their effort.

drive_room_yaw was never emitted. mujoco_shake.py:221 asks for it by name; the exporter only ever made x/y/z. The twist the joint exists for — the thing the code documents as making a far corner move further than the middle — silently never happened, and the 50 t room was left free to rotate under contact torque. Under a shake that previously did nothing, the door now swings 16.1° and the sashes 3–11°, with initial_overlaps: 0.

Chairs were half their weight. OCCUPANCY_DENSITY["chair"] = 14.0 put a task chair at 7.2 kg, and after the room's fit rescaled it, 4.9 kg on the floor against a real 12–15. This is the direct cause of a scene where things fly on contact. Measured over their own bounding box, real chairs cluster at 23–27 kg/m³ regardless of type:

task chair, castors + gas strut   0.49 m³   13 kg -> 26.8
cantilever meeting chair          0.23 m³  5.5 kg -> 23.5
wooden dining chair               0.21 m³  5.0 kg -> 23.4
bar stool                         0.12 m³  5.0 kg -> 41.7

So chair: 25.0, and stool: 40.0 because a stool is small enough that its frame dominates its enclosed volume. A 20 N shove now moves a chair 1 mm rather than 27–52 mm, and 50 N still topples it — correct at 0.66 g against a ~0.55 g tipping threshold.

Also

--polish no longer includes the model-driven QC pass. It was the longest agent pass on the run for the smallest marginal change and nothing downstream reads its output; --quality-pass still asks for it.

scripts/simready/view_scene.py opens an exported scene interactively. MuJoCo's managed viewer fails constructing its window on macOS 26 — it dies the same way on a three-line test model — while the passive viewer is fine, so this drives that and owns the loop. Frozen by default, so a scene can be inspected without settling out from under you.

Caveats

One scan, and a small tidy one — it began with zero interpenetration, which #45's own table shows is not typical. The reporting and chair-density findings generalise; the clean stability result may not.

613 tests pass, ruff clean — and none of them would have caught any of this. grep "drive_room\|actuator" tests/ is empty: the suite tests the sidecar format thoroughly and never asserts on the emitted MJCF's dynamics. That gap is worth closing next, and I have deliberately not done it here.

@ZheningHuang
ZheningHuang changed the base branch from feat/mujoco-sim-scene to main September 9, 2026 19:21
Found running #45 end to end on Office-Elliott from a raw scan. The export
itself held up — 99/99 colliders from sidecars on the seed room, no solver
warnings over 30 s x 8 resets — but three things reported success they had not
earned, and two more meant the compiled physics never reached the simulator.

Reporting

- Reference generation fell back for EVERY object on a missing key and the run
  carried on building from raw evidence sheets. The only signal was
  "0 references ok", which reads as a count. It now names the failure.
- `_object_physics` returned before recording anything when an object had no
  package, so authored fixtures vanished from `export_report.json` and the
  simulate stage's warning — keyed on that list — never fired. The authored
  Office-Elliott room reported an empty `no_sidecar` while 130 of its 229
  colliders had been derived at export. Now recorded, plus `derived_colliders`
  and `sidecar_coverage` so nobody has to subtract two fields.
- The authoring step budget lands gracefully and exits 0 through the same path
  as a finished room, so a truncated room was indistinguishable from a
  complete one. It now says so, and `--author-steps` exposes the budget.

Config

- `GEMINI_API_KEY` and `LR_IMAGE_PROVIDER` were never carried into the
  subprocess that generates references, so the documented Gemini path could
  not work from `.env`. `sanity.py` separately demanded an OpenAI key whatever
  the provider — and would have passed an OpenAI key alongside
  LR_IMAGE_PROVIDER=gemini, then failed at the first object.

Physics

- Every sidecar joint now gets a motor at the effort the asset stated (45 N.m
  that door, 800 N that desk lift). A motor rather than a position servo on
  purpose: a servo would clamp every door shut at ctrl=0 and stop a hung door
  swinging under the shake. Passive drift is unchanged; each joint now reaches
  100% of its range when driven.
- `drive_room_yaw` was asked for by `mujoco_shake` and never emitted, so the
  twist was a silent no-op and the room was free to rotate under contact
  torque. The door now swings 16.1 deg under a shake that did nothing before.
- Chair occupancy density 14.0 -> 25.0 kg/m3 (stool 40.0). 14.0 put a task
  chair at 4.9 kg against a real 12-15; measured over their own bounding box
  real chairs are 23-27 whatever kind they are. A 20 N shove now moves a chair
  1 mm rather than 27-52, and it still topples at 50 N, which is correct.

QC is no longer part of `--polish`. It is the longest agent pass on a run for
the smallest marginal change and nothing downstream reads it; `--quality-pass`
still asks for it.

`scripts/simready/view_scene.py` opens an exported scene interactively.
MuJoCo's managed viewer fails constructing its window on macOS 26 — including
on a three-line test model — so this drives the passive viewer instead, frozen
by default so a scene can be inspected without it settling out from under you.

613 tests pass, ruff clean. None of them would have caught any of this:
`grep "drive_room\|actuator" tests/` is empty.
…loads clean

Found by running the pipeline with `--author-steps 20`. A short authoring
session wrote `Skirting0` with `rests_on: Floor0` and no `attached_to`, and
`rests_on` short-circuits every other test — so 86 kg of trim ringing the room
was emitted as a FREE body, 40 mm inside the door lining. MuJoCo read the
overlap as stored energy and threw it 158 mm; the scene failed the stability
gate on that one body. The fully authored room escaped only because the author
got as far as writing `attached_to: Room_Shell`, which is to say the room was
one interrupted session away from unusable with nothing to warn anyone.

Two changes, one specific and one general.

TRIM, alongside the existing SCENERY set: skirting, trunking, coving,
architrave and the rest are nailed to the fabric and run inside the walls they
trim. They are never free bodies, and unlike SCENERY that holds at any height —
trunking sits at 0.95 m. Pinned whatever the layout claims, and reported as
`trim_pinned`.

And the backstop that would have caught it whatever it was called: the export
now compiles the file it just wrote, steps it once, and records whether it
starts clean. `loads_clean` and `initial_overlaps` go in the report and the
simulate stage refuses to call a scene that starts interpenetrating a quiet
success. Best-effort and non-fatal — a scene that cannot be loaded is still
written, because a file you can inspect beats no file at all.

Office-Elliott authored at 20 steps: FAIL (22 contacts, -39.6 mm, ejected
158 mm) -> PASS (0 contacts, 0.0 mm, no warnings, nothing escaped). The fully
authored room re-exports unchanged and also reports `loads_clean: true`.
@ZheningHuang
ZheningHuang force-pushed the fix/simready-honest-reporting branch from d97c04c to 976ae3e Compare September 9, 2026 19:21
@ZheningHuang
ZheningHuang merged commit cc997e8 into main Sep 9, 2026
1 check passed
@ZheningHuang
ZheningHuang deleted the fix/simready-honest-reporting branch September 9, 2026 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant