Skip to content

[DEMO] WrenchComposer zero-range regression on perf-smoke (RED)#9

Open
Neil4561 wants to merge 2 commits into
unified-POCfrom
neil/perf-demo-wrench-regression-v2
Open

[DEMO] WrenchComposer zero-range regression on perf-smoke (RED)#9
Neil4561 wants to merge 2 commits into
unified-POCfrom
neil/perf-demo-wrench-regression-v2

Conversation

@Neil4561

@Neil4561 Neil4561 commented Jun 25, 2026

Copy link
Copy Markdown

Demo PR (not for merge)

RED case for the performance smoke-test demo.

Change: reintroduces the WrenchComposer regression from upstream isaac-sim#5265.

Explanation: some environments configure an external force event with force and torque set to zero. That means "do not actually push the robot." The normal code sees that everything is zero and skips the work. This PR removes that shortcut, so the simulator still goes through the WrenchComposer path to write zero forces and zero torques into the wrench buffers. The physical result is the same, but the simulator does extra bookkeeping work, which shows up as lower FPS on affected tasks.

Isaac-Velocity-Flat-G1-v0 is the important affected task in the demo because it exercises that external-wrench path. Cartpole and Factory act as controls: they stay near baseline, showing that the regression is not a global slowdown across every benchmark cell.

Matrix: Isaac-Cartpole-Direct + Isaac-Factory-GearMesh-Direct-v0 (physx) + Isaac-Velocity-Flat-G1-v0.

Trigger via mirror to pull-request/<N> or workflow_dispatch.

Neil4561 added 2 commits June 25, 2026 18:53
…m#5265

Remove the all-zero early-return guard in apply_external_force_torque so
every step samples and sets zero-magnitude forces through the
WrenchComposer path instead of short-circuiting. Reproduces upstream
isaac-sim#5265 (~throughput hit on locomotion/velocity tasks whose
external-force event uses all-zero ranges, e.g. Isaac-Velocity-Flat-G1-v0).
Demo only; not for merge.
Trim to three well-baselined demo tasks (Cartpole physx/newton, Factory
physx, G1 physx/newton) for a clean multi-row verdict table. Demo only;
not for merge.
@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown

Performance Smoke Results

Baseline read SHA: e87de2efea41

Run context

  • GPU: NVIDIA RTX PRO 6000 Blackwell Server Edition
  • Runtime: cuda=12.8, driver=595.71.05, warp=1.13.0
  • Threshold policy: rolling baseline per row. Each task/backend is compared against its own compatible baseline samples using median/MAD.

How to read this table

  • PASS: no meaningful slowdown was detected.
  • WARN: suspicious or uncertain result, such as insufficient baselines, retry-only success, or a slowdown in the WARN band.
  • BLOCK: a blocking-level regression signal was detected. In this POC, BLOCK is advisory (it flags a regression) and only fails the check when the gate is explicitly configured as blocking.
  • HARD_FAILURE: the benchmark did not produce usable FPS data, for example an import/init/runtime failure or config mismatch.
  • Delta: change in FPS versus the baseline. + means faster than baseline (speedup); - means slower than baseline (slowdown).
  • Threshold policy: explains which comparison rule produced the verdict. rolling_window means each row uses its own compatible baseline median/MAD window, not one shared FPS cutoff.
  • Noise: baseline MAD as a percent of baseline FPS. Higher noise means the task/backend naturally varies more from run to run.
  • Phase: the stage a HARD_FAILURE happened in (e.g. import, init, runtime); blank when the benchmark ran to completion.
  • Retried: yes if the cell only passed after an automatic re-run.
Task Backend Verdict FPS Baseline Delta (+ faster / - slower) Noise Samples Phase Notes Retried
Isaac-Cartpole-Direct newton PASS 1020799.6 1010818.0 +0.99% 0.80% 8 p99/med=1.174; outliers=0 no
Isaac-Cartpole-Direct physx PASS 288676.8 274577.2 +5.14% 1.98% 8 p99/med=1.2; outliers=0 no
Isaac-Factory-GearMesh-Direct-v0 physx PASS 1209.0 1199.0 +0.84% 1.48% 6 p99/med=1.03; outliers=1 no
Isaac-Velocity-Flat-G1-v0 newton BLOCK 29527.6 33058.7 -10.68% 1.70% 6 p99/med=1.113; outliers=0 no
Isaac-Velocity-Flat-G1-v0 physx BLOCK 8790.6 10962.8 -19.81% 2.71% 6 p99/med=1.097; outliers=0 no

Updated for 33bd4b2 · run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant