Skip to content

Fix example demos: tensor handling, keyboard teleop #1952

Open
kmy17518 wants to merge 7 commits intoStanfordVL:mainfrom
kmy17518:fix/examples
Open

Fix example demos: tensor handling, keyboard teleop #1952
kmy17518 wants to merge 7 commits intoStanfordVL:mainfrom
kmy17518:fix/examples

Conversation

@kmy17518
Copy link

Summary

Addresses issues in the following example demos:

  • BEHAVIOR Task Demo
    python -m omnigibson.examples.environments.behavior_env_demo

  • Scene Tour Demo
    python -m omnigibson.examples.scenes.scene_tour_demo

  • Robot Control Demo
    python -m omnigibson.examples.robots.robot_control_example


Changes

  1. Constrain JoyLo dependencies

    • numpy
    • opencv-contrib-python
  2. Fix tensor conversions in CameraMover

  3. Fix keyboard teleop event handling

    • Prevent key release from clearing action before it's used
  4. Fix typo in print_keyboard_teleop_info

    • Corrected i/k and j/l key mappings

Copilot AI review requested due to automatic review settings January 23, 2026 08:57
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes multiple example demo issues related to camera image / video recording, keyboard teleoperation behavior, and JoyLo dependency compatibility with Isaac Sim.

Changes:

  • Add pip constraints when installing JoyLo to avoid incompatible numpy / opencv-contrib-python versions.
  • Fix CameraMover recording paths by converting torch tensors to CPU numpy arrays, and make spline-derived values explicitly numeric.
  • Adjust keyboard teleop release handling and correct printed key mapping for differential-drive control.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
setup.sh Adds constrained dependency installation for JoyLo to maintain OmniGibson/Isaac Sim compatibility.
OmniGibson/omnigibson/utils/ui_utils.py Fixes CameraMover tensor-to-numpy conversions for image/video recording and updates keyboard teleop behavior + printed key mappings.
OmniGibson/omnigibson/examples/scenes/scene_tour_demo.py Updates waypoint tensor creation for camera trajectory recording in the scene tour demo.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 374 to 380
# Constrain deps to keep OmniGibson / Isaac Sim compatible
JOYLO_CONSTRAINTS_FILE="/tmp/joylo-constraints.txt"
cat > "$JOYLO_CONSTRAINTS_FILE" << 'EOF'
opencv-contrib-python<=4.11.0.86
numpy<2
EOF
pip install -e "$WORKDIR/joylo" -c "$JOYLO_CONSTRAINTS_FILE"
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using a fixed constraints file path in /tmp can collide when multiple installs run concurrently (and leaves a stale file behind). Prefer creating the constraints file with mktemp and removing it afterwards (e.g., via a trap) so the install is race-free and self-cleaning.

Copilot uses AI. Check for mistakes.
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in f5b0eae

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you directly add numpy<2 and opencv-contrib-python<=4.11.0.86 to the joylo setup.py file? I think this should work so we don't have to modify the main setup.sh file.

Also what's the reason for constraining opencv-contrib-python<=4.11.0.86?

Copy link
Contributor

@wensi-ai wensi-ai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm overall, please see comments above

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants