Skip to content

Conversation

@xyao-nv
Copy link
Collaborator

@xyao-nv xyao-nv commented Dec 18, 2025

Summary

Users may want to render different textures for same objects in parallel envs, especially for USDs they don't maintain/have source access. This is a step towards heterogeneous parallel envs, i.e. same obj, diff textures.

Detailed description

  • There are obvious drawbacks from adding all kinds of rendering styles directly into USD files, and selecting those variants using USDFileCfg variants params. E.g. USDs will get bigger and bigger. And maybe there are USDs you don't maintain.
  • As a consumer of those USDs, it creates the material (default/rand) on the fly, and wraps the assets with those material variants. Following guidances from IsaacSim
  • It is only texture here, aka rendering related. PhyX related properties will be tricky to do, like scaling due to the collider.
  • Users specify the object names to add those variants, and it fetches the prim paths for all those objects in policy_runner.py

E.g. kitchen cracker box
Screenshot from 2025-12-17 23-56-06

Screenshot from 2025-12-18 00-29-03 Screenshot from 2025-12-18 00-30-07

Copy link
Collaborator

@alexmillane alexmillane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for looking at this. The result looks cool.

My main concern is that we're doing the randomization at the top-level in the policy runner. This falls afoul of the single responsibility principle. The policy-runner is now concerned with running the policy and implementing a (growing list of) scene variations.

Can we somehow roll this type of variation into the objects themselves? Perhaps we could activate color variations per object with an API like object.activate_color_variations().

I notice that isaac lab has some events for doing stuff like this, for example randomize_visual_color. Is there some reason we can't use this event? Perhaps object.activate_color_variations() could trigger the addition of such an event to the IsaacLabArenaManagerBasedRLEnvCfg? Then we wouldn't see this stuff at the top level in the policy runner. We would just build our scene out of objects, with optional variations applied, and run the policy as usual?

Comment on lines +38 to +49
# Post-spawn injection
if args_cli.randomize_object_texture_names is not None and len(args_cli.randomize_object_texture_names) > 0:
from isaaclab.sim.utils import get_current_stage

from isaaclab_arena.utils.usd_helpers import randomize_objects_texture

randomize_objects_texture(
object_names=args_cli.randomize_object_texture_names,
num_envs=args_cli.num_envs,
env_ns=env.scene.env_ns,
stage=get_current_stage(),
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my opinion, we need to keep variations (and other feature requests) out of the policy_runner.

We should aim for an interface where objects can have texture variations applied. Something like:

apple.apply_color_variations()

That way, the policy runner doesn't need to worry about this stuff; it's all hidden away in the object definition and the policy runner just runs the policy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants