[Question] How to create a clickable button in Isaac Sim when using XR

[sashansamarajeewa]

Build Info

Describe the versions that you are currently using:

• Isaac Lab Version: 2.3.0
• Isaac Sim Version: 5.1.0

Description

I’m using Isaac Sim with XR and I’d like to create an in-world button that the XR user can press to trigger Python callbacks (e.g. start/stop).

I’ve tried using omni.ui widgets with WidgetComponent and UiContainer,

class SimpleButtonWidget(ui.Widget):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        with ui.ZStack():
            with ui.VStack():
                self._btn = ui.Button("Start", clicked_fn=self._on_click)

    def _on_click(self):
        print("button clicked") 

def create_xr_button_panel():
    def on_constructed(widget_instance):
        pass

    widget_component = WidgetComponent(
        SimpleButtonWidget,
        width=0.3,
        height=0.1,
        resolution_scale=20,
        unit_to_pixel_scale=30,
        update_policy=sc.Widget.UpdatePolicy.ALWAYS,
        construct_callback=on_constructed,
    )

    space_stack = [
        SpatialSource.new_translation_source(Gf.Vec3d(0.3, 0.9, 1.5)),
        SpatialSource.new_rotation_source(Gf.Vec3d(
            math.radians(90), 0.0, 0.0,
        )),
    ]

    ui_container = UiContainer(widget_component, space_stack=space_stack)
    return ui_container

The widget shows up correctly in XR, but the button is not clickable from inside XR.

What is the recommended/supported way to create interactive buttons that can be pressed by the XR user inside Isaac Sim? If there is an example or sample extension that shows a clickable XR button, could you point me to it?

Thank you