Add heterogeneous clone scene composition#6529
Conversation
Greptile SummaryThis PR introduces heterogeneous clone-scene composition: a new
Confidence Score: 3/5Two real concerns need resolution before merging: the mutable-list default on The composition engine itself (
Important Files Changed
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if len(asset_names) != len(variant_counts): | ||
| raise ValueError(f"Expected one variant count per asset, got {len(variant_counts)} and {len(asset_names)}.") | ||
| if not asset_names: | ||
| raise ValueError("Expected at least one asset name.") | ||
| if any(count <= 0 for count in variant_counts): | ||
| raise ValueError("Variant counts must be positive.") | ||
|
|
||
| if not clone_combinations: | ||
| rows = itertools.product(*[range(count) for count in variant_counts]) | ||
| return torch.tensor(list(rows), dtype=torch.long, device=device) | ||
|
|
There was a problem hiding this comment.
InclusionSet(assets=[]) silently produces all-inactive rows
make_valid_clone_combinations does not validate that each InclusionSet.assets list is non-empty. An entry like InclusionSet(assets=[]) is accepted: combination_assets for that row becomes an empty intersection, and the loop marks every claimed asset as inactive, producing a valid-looking tensor row of all (-1,) entries. The result is a set of environments that receive none of the task-specific assets — an empty env that is almost certainly unintended and produces no validation error.
Consider adding if not combination.assets: raise ValueError(...) before the intersection step.
| def _make_scene( | ||
| source: InteractiveSceneCfg, | ||
| entities: dict[str, AssetBaseCfg], | ||
| slots: dict[str, str], | ||
| rows: list[InclusionSet], | ||
| ) -> InteractiveSceneCfg: | ||
| """Build a configclass that InteractiveScene can parse normally.""" | ||
| clone_cfg = copy.deepcopy(source.clone_cfg) | ||
| clone_cfg.clone_combinations = rows | ||
| values = {name: copy.deepcopy(getattr(source, name)) for name in InteractiveSceneCfg.__dataclass_fields__} | ||
| values["clone_cfg"] = clone_cfg | ||
| namespace = {"__module__": __name__, "__doc__": "Scene configuration produced by scene_add.", **entities} | ||
| scene_type = configclass(type("_AddedSceneCfg", (InteractiveSceneCfg,), namespace)) | ||
| scene_cfg = scene_type(**values) | ||
| setattr(scene_type, _SLOT_METADATA, slots) | ||
| return scene_cfg |
There was a problem hiding this comment.
_AddedSceneCfg class is not picklable
type("_AddedSceneCfg", (InteractiveSceneCfg,), namespace) creates a fresh class on every scene_add call. Even though __module__ is set to scene_composition.__name__, the class is not actually importable from that module — pickle looks up _AddedSceneCfg in isaaclab.scene.scene_composition at deserialisation time and raises AttributeError. Any code path that tries to serialise a composed scene config (e.g., checkpointing, Ray workers, multiprocessing spawn mode) will fail silently or with a confusing error. Consider registering the produced type in the module dict (sys.modules[__name__]._AddedSceneCfg = scene_type) or documenting this constraint in the docstring.
d9819f7 to
4d8df66
Compare
4d8df66 to
78b2505
Compare
# Description
Replaces expanded `/World/envs/env_.*` prim paths with the canonical
`{ENV_REGEX_NS}` macro across task scene configurations, declares the
Franka Cabinet robot and end-effector frame in the scene configuration
instead of mutating them after construction, and aligns the Reach and
OpenArm Lift table bindings and initial states.
These are behavior-preserving config normalizations (`{ENV_REGEX_NS}`
expands to the same regex at scene construction). They are split out of
#6529, which composes task scenes and requires the canonical
literal-binding form.
## Type of change
- This change requires a documentation update (changelog fragment
included)
## Checklist
- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have added a changelog fragment under
`source/<pkg>/changelog.d/`
- [x] My name is in the `CONTRIBUTORS.md`
🤖 Generated with [Claude Code](https://claude.com/claude-code)
a6e5a33 to
16054a6
Compare
github-actions
Bot
added
documentation
|
This PR is missing scripts/demos/multitask_clone_scene.py which is reported in the PR description! |
Enabling fabric at runtime (physx_manager._load_fabric) re-solves the full extension graph, which fails on builds where an enabled extension has locally-unresolvable dependencies. Loading it with the app avoids the re-solve entirely.
Add a showroom entry and screenshot for the heterogeneous scene demo, and fix the demo docstring usage examples that still referenced the old multitask_clone_scene.py path.
This reverts commit 0803b7e.
- test_collect_asset_cfgs_resolves_env_regex_macros: the test bypasses __init__ via object.__new__, so _env_ns was never set; populate _env_regex_ns and _env_ns from CloneCfg to match what __init__ does - test_usd_replicate_self_copy_skips_copy_spec: test patched isaaclab.cloner.cloner_utils.Sdf, but Sdf is imported only in isaaclab.cloner.usd; fix the import target - test_resolve_clone_plan_source_partial_coverage_raises: the PR updated resolve_clone_plan_source to allow partial-env coverage (heterogeneous scenes); update the test to assert the return value instead of expecting NotImplementedError
e7ee838 to
e66e04d
Compare
The kitless-factories test forbids isaaclab.assets during config construction, but the scene composition helpers imported AssetBaseCfg at module scope. Defer the import to the composition function that needs it for isinstance checks; annotations resolve lazily.
2 participants
Important
Depends on #6534 (task scene-binding normalization). Merge #6534 first — this PR's task-side diffs assume the canonical
{ENV_REGEX_NS}bindings introduced there.Summary
InteractiveScenecan instantiate different task asset sets across different environment rows.scene_addcomposition utilities for folding registered task scene configs into a single declarative scene with clone combinations.scripts/demos/multitask_clone_scene.py, which composes supported task scenes, skips global floor/light assets, and runs the resulting scene directly without task envs or managers.AssetBaseCfgto kinematicRigidObjectCfg, so they participate in normal physics/USD replication instead of needing scene-level specialMotivation
This PR is meant to support multitask scene demos where each cloned environment may contain a different robot/task asset bundle. The previous clone flow assumed every env received the same source assets, which made it hard to build one scene from multiple existing task configs without manually rewriting each task into a shared schema.
The composition path keeps the existing task scene configs as the source of truth. It reads their declarative scene fields, deduplicates equivalent asset slots, and emits clone-combination rows that the cloner can use to populate only the assets needed by each task group.
Design Notes
scene_addonly composes env-scoped, spawned, physics-backed assets rooted at literal{ENV_REGEX_NS}/Leafpaths. Global floors/lights should be skipped during composition and added once to the composed scene.AssetBaseCfgis rejected inscene_add; env-local props that need per-env instances should be modeled asRigidObjectCfgorArticulationCfginstead.Test Plan
./isaaclab.sh -p -m py_compile source/isaaclab/isaaclab/scene/interactive_scene.py source/isaaclab/isaaclab/scene/scene_composition.py source/isaaclab_tasks/isaaclab_tasks/core/lift/lift_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/core/lift/config/franka_soft/franka_soft_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/core/reach/reach_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/stack/stack_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/stack/stack_instance_randomize_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/deploy/reach/reach_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/deploy/gear_assembly/gear_assembly_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/lift/config/openarm/lift_openarm_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/reach/config/openarm/unimanual/reach_openarm_uni_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/core/lift/config/franka_soft/franka_cloth_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/assemble_trocar/g129_dex3_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/place/config/agibot/place_toy2box_rmp_rel_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/place/config/agibot/place_upright_mug_rmp_rel_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/pick_place/nutpour_gr1t2_base_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/pick_place/exhaustpipe_gr1t2_base_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/locomanip_pick_place/locomanipulation_g1_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/locomanip_pick_place/fixed_base_upper_body_ik_g1_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/pick_place/pickplace_gr1t2_env_cfg.py source/isaaclab_tasks/isaaclab_tasks/contrib/pick_place/pickplace_unitree_g1_inspire_hand_env_cfg.pyAssetBaseCfgassignments after converting static props toRigidObjectCfg.