Domino system-ID tasks: reach-limited min-block (friction) + heavy-block obstacle (mass)#44
Merged
Merged
Conversation
…et goals When set, the callable is the environment-side success criterion and BaseEnv.goal_reached evaluates it in place of the plain goal-atom check (which remains the None default). Propagated through the PyBullet task conversion and alt-goal replacement. Deliberately not copied into the agent-facing Task, so approaches cannot peek at the ground-truth criterion. First user: the domino min-block reward (topple AND at most K* movable blues used).
…ation arms Settings: domino min-block task mode (reach-limited start/target spans, per-task K* budget via MinBlockReward), domino_true_friction vs domino_planning_friction (planning base sims run at the miscalibrated value), task cache dir under saved_datasets/, and agent_sim_learn_oracle_sim_params extended to also grant planning base sims the TRUE physical params (absorbs the former agent_planner_oracle_base_sim_params; the agent_ prefix keeps it outside the task-cache key on purpose). agents.yaml: baseline (agent_base_sim_no_learning) and oracle (agent_oracle_hybrid_sim) both run agent_sim_learning and differ in exactly that one flag, so the arms are a clean one-bit ablation of base-sim calibration. envs/all.yaml: min-block domino config (true 0.1 vs planning 0.5, span band, 4 staged blues); old chain-task config kept commented for easy revert.
…r search, per-leg differentiation The min-block pipeline lives in task_generators/min_block_generation.py (quota loop, disk cache keyed by config+seed+source digest) with the layout/probe machinery in task_generators/min_block_utils.py; the env keeps the physics and reward semantics (MinBlockReward, per-instance set_domino_physical_params via changeDynamics, planning-friction wiring for skip_process_dynamics envs). K* is always a simulated minimum with the real Push at the agent-typical params [0.04, 0.05] (a stronger probe push previously masked the miscalibration the tasks exist to expose). Straight tasks: evenly-spaced chains, kept only when the planning-friction recount is strictly cheaper. Turn tasks: K* searches only agent-buildable layouts -- straight-line probes plus ONE natural-yaw corner blue with sim-calibrated configs (derived from the oracle run's own -36deg corner solution); the generator's mirrored 45deg pair is deliberately excluded since no planner would propose it. Because natural corners barely propagate at the planning friction, turn differentiation uses a relaxed per-LEG certificate: straight-chain probes of each leg at both frictions, kept only when the wrong model under-counts the legs (the corner cost cancels). Empirical notes: corner viability at the two frictions is anti-correlated across every domain lever (mass, restitution, spinning/rolling friction, shape), so the knife-edge corner search is structural, not a tuning miss.
Self-contained reveal.js deck (motivation -> pipeline overview -> reward, anatomy, K* search, corner-layout search, turn-physics A/B, differentiation filter, per-task examples). Figures are generated by checked-in scripts that drive the real task-gen code: the corner-candidate grid, the turn yaw A/B (scoped: natural alignment fails only at min-block gaps; the legacy generator's natural corners work at tighter gaps), and the five live test tasks with calibrated vs miscalibrated solutions, every panel sim-verified.
…ve obsolete comments for clarity
…-first turn filtering, partial-cache flagging Generation took ~20 min per arm (vs ~2 min for the old straight-only generator): 29 of 33 failed turn attempts paid for the full corner layout search (dozens of robot Push rollouts each) before rejection, identical leg spans were re-probed on every attempt, and both arms regenerated in parallel after a cache-key miss. - Memoize straight_span_k_star per (1cm span bucket, budget, physics override), cleared once per generation run. - _make_turn_task: run the cheap memoized leg probes BEFORE the corner layout search — a feasibility bound (legs' straight-chain minima already exceed the staged budget) and the per-leg differentiation certificate now reject most attempts for free. - Cache format now records num_requested; a partial set (quota loop hit its attempt cap) warns loudly on every reload instead of silently shrinking the eval. - Exclude render-only flags (camera width/height, draw_debug) from the cache key so a resolution change no longer orphans a generation cache.
…_block_tasks Each task is a dogleg: a gray domino-shaped block with true mass 1000 kg (DominoComponent.heavy_block_true_mass) stands on the start's fall line, yawed ~30 degrees off it, posing as a ready-made bend link toward the target. Planning sims believe it has normal domino mass (new heavy_block_mass physical-param override, set at env init for skip-dynamics envs), so an uncalibrated planner runs its chain into the gray block and dies there; a mass-calibrated planner bends early with its own corner blue and cuts across. Generation (reusing the min-block quota loop / cache / MinBlockReward): - heavy_dogleg_k_star probe (all entry/exit splits, swept exit gaps, memoized at the canonical anchor); - certificate: believed dogleg exists, believed detour is no cheaper (both at planning physics), true dogleg dead, true detour K* within the staged blues (K* may equal them: differentiation is by topple failure, not the over-build cap); budget = detour K*; - shape (legs/bend from probe-sweep bands, lattice-snapped for memo reuse) certified once, placement retried across start poses; - compute_turn_k_star gains an extra-scene arg with spawn-overlap pruning so detour probes include the gray obstacle. Support changes: reset_state moves absent dominoes out of view by identity (heavy tasks use the last slot, breaking the old prefix assumption), gray blocks get true mass at every reset with the generic mass override shielded, staging exempts gray blocks, place_domino grows is_heavy_block.
…-at-corner turn) Replaces the dogleg geometry with two variants that keep natural alignments, mixed per domino_min_block_turn_ratio: - straight: start -> gray -> target collinear and co-facing; the believed (normal-gray-mass) plan chains straight through the gray as a free link; the true solution is a half-circle swerve around it (new swerve layout family in min_block_utils: heading profile phi*sin(2*pi*t), aligned at both ends, knocks within the measured ~33-degree propagation tolerance at phi 25-40 degrees); - turn: find the believed-cheapest L corner layout at the family minimum (all candidates of the cheapest toppling k are scanned, not just the first), and stand the gray exactly at that natural corner pose. The lure (gray as free corner) is structurally one blue cheaper than every own-corner alternative; the true solution skips around via the existing corner search with the gray as obstacle. Both certify per task: believed lure exists, lure dead at the true physics, true swerve/detour K* within the staged blues; budget = K*. Lure probes and corner blueprints memoize at the canonical anchor; placement retries reuse the certificate. The turn-variant lure needs believed friction == true friction (corners never propagate at 0.5), so the domino_heavy launch config drops domino_planning_friction: this env isolates the MASS dimension. Also: launch config block (envs/all.yaml domino_heavy + agents.yaml override; min-block block kept with SKIP), heavy task-examples slide + figure script in the deck, slide text updated.
The 2026-07-03 oracle run shipped a 4/5 PARTIAL set: turn attempts survive ~1-in-30 (the per-leg differentiation certificate's dead band dominates — 23 of 33 drops), and the old 12x+20 cap gave only 80 attempts. Rejected attempts are nearly free since the probe memo, so the cap is now 36x+40. Turn legs also snap to the 1 cm probe-memo lattice: repeats are guaranteed memo hits (no bucket-edge double probes) and the drop log becomes a readable coverage map over the finite (entry, exit) cells for any future band retuning. Validated by regenerating the seed-0 friction sets with the exact launch flags: 5/5 test tasks (2 turns K*=3, 3 straights K*=1, 37 turn drops absorbed) in ~10 min; train 1/1; straight-path smoke unchanged.
Per-attempt cost was dominated by three hot spots, all fixed: - placement loops now stage FIRST (pure geometry) and only pay the sim re-verification (real-pose swerve / detour search) on a staged pose — staging failures on the single-row grid were re-running full searches per pose try; - the true-dead certificate scans only the believed cost k_bel (new only_k arg on heavy_dogleg_k_star): a block-minimizing planner only builds believed-cheapest layouts, so other counts cannot leak; - the believed corner-blueprint sweep is capped at the chord's straight-chain minimum + 2 (memoized probe): shapes where no corner propagates used to sweep every candidate at every k (~minutes) before concluding None. Blueprints are also computed for one side and mirrored for the other (physics is mirror-symmetric; the gray-substituted lure rollout re-verifies the mirrored geometry). Smoke: straight task kept in 8s (was 60s), turn attempts ~10-13s flat (was up to 60s+ with multi-minute blueprint misses), full single-flag generation 41s (was 227s); scene/physics/cache assertions unchanged.
…e push The same layout alternated topple/no-topple on repeated probe rollouts: _set_state reaches the robot pose by IK FROM THE CURRENT arm configuration, so consecutive probes inherited ~1e-3 rad wrist differences from wherever the previous rollout parked the arm — enough to flip knife-edge (corner/swerve) layouts. Symptoms: the heavy turn pipeline contradicted itself (blueprints blessed by a lucky flicker, then the physically identical gray-substituted lure failing 71% of re-rolls), and figure scripts reported certified solutions as 'not reproducible'. _layout_topples and _chain_topples now set the arm to its exact initial joints after state assembly. Probes are fully deterministic and match episode-start conditions (episodes also begin from the initial joints). Verified: 6x identical repeats now agree, and motion-planning ON vs OFF give identical outcomes once pinned — the flag never changed push physics; the drift did.
Diagnosis from the regenerated figure: with deterministic probes, one task's recorded detour K*=3 re-derived at k=4 in a fresh process (and another's K*=4 at k=2) — knife-edge corner layouts are sensitive to the simulator's contact-solver history even with pinned arm joints, so an exact-K* budget recorded under generation's sim history can be unreachable under execution's. Heavy tasks differentiate on topple-vs-not (the believing planner dies at the gray regardless of budget), so MinBlockReward's max_blocks is now the STAGED blue count: borderline-K* flips can no longer make a task unsolvable-within-budget, for the agent or for figure re-derivation. The swerve/detour searches remain as solvability certificates (a within-staged-blues solution exists). Docs, figure labels (budget=N), and settings comment updated.
Task 1 of the seed-0 set kept failing figure re-derivation even with the staged-blues budget: its detour family had exactly ONE toppling layout out of 83 candidates, and knife-edge single layouts flip with the simulator's contact-solver history — a task whose only solution toppled once during generation can be unsolvable under a fresh sim. compute_turn_k_star and swerve_k_star gain a min_hits arg (scan continues until that many distinct layouts have toppled across k <= budget); both heavy makers demand min_hits=2 at the real-pose check, so shipped tasks always have at least two independent solutions and survive history variance.
The reset path repositions bodies via resetBasePositionAndOrientation, which does NOT clear velocities — a domino that was mid-fall when the previous rollout ended carried its momentum into the next 'static' scene. This made probe outcomes depend on what the simulator ran beforehand: the same layout toppled or died based on sim history, which broke figure re-derivation of certified solutions and would equally have broken execution-time reproducibility of generation certificates. (deterministicOverlappingPairs was tested and is NOT the cause.) DominoComponent.reset_state now zeroes every domino body's linear and angular velocity. Scramble test: a marginal corner layout re-probed after batches of unrelated rollouts now agrees 4/4 (was flipping).
… a fresh sim The turn lure (gray-substituted corner layout) was certified only at the canonical anchor and transformed to the task's real pose — but pose transfer is only approximately physics-preserving, and a shipped task's real-pose lure turned out never to topple believedly (0 of 9 corner-matching layouts): the baseline would not even be lured. The placement loop now re-verifies the transformed lure at the real pose (believed-topple + true-dead, 2 rollouts, after free staging) and retries the pose on failure. The examples figure also re-creates the simulator between generation and panel derivation: a certified detour that re-derives cleanly in a fresh process (2 hits at k=2) failed inside the long-lived generation process — residual sim context still shifts knife-edge outcomes, and a fresh simulator is also the faithful (execution-like) context.
The real-pose lure re-verification lowered turn placement yield (a certified shape now also needs a pose where the transferred lure physically works) and the seed-0 set came out 4/5. Failed poses are cheap (~2 rollouts), so the pose-retry cap doubles. Figure: object roles are matched by equality instead of identity — the env is re-created between loading and panel derivation, so family-derived layouts are keyed by the fresh env's objects (equal by name to the cached tasks').
…ebuild All 15 panels re-derive and sim-verify in a fresh simulator: two turn tasks (gray at the natural corner; skip-around detours topple, believed gray-corner plans die at the gray) and three straights (half-circle swerves topple, believed through-gray chains die). Deck rebuilt with the heavy-block intro and examples slides.
Both the envs/all.yaml block and the agents.yaml excluded_predicates override are commented out together (a FLAGS-only override without its base env breaks the launcher). The min-block domino block stays present with SKIP: True. Uncomment the domino_heavy pair to launch the heavy-block arms; the task cache in saved_datasets is prewarmed at the current code digest.
… lint fixes - run_autoformat.sh (docformatter reflows) applied to the fig scripts and domino task generators; verified stable under the CI-pinned tool versions (yapf 0.32.0, isort 5.10.1, docformatter 1.4). - docs/ added to the mypy exclude and pylint ignore-paths: it holds one-off slide/figure-generation scripts, not library code (both configs already excluded asset directories). - Lint fixes in the domino code: unused imports, demo-block variable shadowing in env.py's __main__, an over-long docstring line, and the real-pose lure probe now binds its layout via functools.partial (cell-var-from-loop + typing clean). Verified locally: mypy . (native and --platform linux) clean over 610 files; pylint 10.00/10 on every changed file; autoformat idempotent; 55/56 branch-relevant unit tests pass — the one failure (test_push_second_switch_boil_position_mode) fails identically on master locally (known macOS-vs-Linux switch-push divergence; passes on CI Linux).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
Two task families for the domino base-sim system-identification experiments, sharing one generation pipeline (quota loop, simulated certificates, disk cache keyed on config + seed + source digest):
Min-block tasks (friction dimension) —
domino_min_block_tasksMinBlockReward(newEnvironmentTask.reward_fn) with budget K* = the simulated minimum blues at the true friction.Heavy-block tasks (mass dimension) —
domino_heavy_block_tasksheavy_block_massphysical-param override), so the believed-cheapest plan runs through the gray block and dies against it at execution; the true solutions are a half-circle swerve around it / a skip-around detour with an own corner.Determinism fixes (prerequisites for all of the above)
Also
docs/envs/domino_min_block_task_gen_slides.html) documenting motivation, pipeline, K* search, differentiation filters, and sim-verified task examples for both families.scripts/configs/predicatorv3/); the heavy env block is present but commented out until those arms launch.Preliminary experiment
python scripts/local/launch_simp.py -c predicatorv3/agents.yaml(seed 0, min-block tasks, true friction 0.1 vs planning friction 0.5):agent_oracle_hybrid_sim(GT params)agent_base_sim_no_learningThe oracle solves the min-domino tasks while the friction-mismatched baseline fails: it over-estimates topple reach, under-builds, and its chains die short of the target — exactly the miscalibration signal the sysID learner is meant to recover.