Request for feedback from the IsaacLab community

[jsmith-bdai]

Hello IsaacLab Community,

Over the past few years, IsaacLab (and the previous Orbit framework) have seen incredible growth. We believe now is a great time to gather valuable feedback from the community in one place. Your contributions, whether through questions, pull requests, bug reports, or sharing your unique use cases, have been instrumental in this progress. We're thrilled to see the community expand and thrive!

We'd love to hear your thoughts, and all feedback is welcome! Here are a few areas to consider when sharing:

1. Has the organization and structure of code been easy to understand? (1-10, 1='What structure?', 10='The structure was very intuitive and easy to grasp')
2. Are there any features or integrations you would like to see prioritized in upcoming releases?
3. How would you rate the support provided through GitHub issues and discussions? (1-10,1 ='Support?!, 10='Stop reading my mind!')
4. Would you like to see more open-source projects that allow for your contributions?
5. How easy was it to get started with the framework? (1-10, 1='I spent months before understanding basics', 10-'I understood everything in an hour'). Were there any particular pain points during installation or initial setup?
6. Have the examples and tutorials provided been helpful for understanding key concepts? What additional examples or tutorials would you like to see?
7. Have you encountered any performance issues or bottlenecks? If so, how could these be improved?
8. How does the framework compare to other tools or libraries you've used for similar purposes? What other libraries have you used for your RL needs?
9. Anything else you can think of?

Your feedback will play a crucial role in shaping the future development of the framework and help us prioritize resources to better serve this growing community.

Thank you for your continued support!

Best regards,
The IsaacLab Team

[johnnylu305]

Dear IsaacLab Community,

I would like to thank the IsaacLab team for their work. Below is my feedback.

1. Has the organization and structure of code been easy to understand? (1-10, 1='What structure?', 10='The structure was very intuitive and easy to grasp')

Ans: 8

2. Are there any features or integrations you would like to see prioritized in upcoming releases?

Ans: My project involves flying a drone equipped with an RGB-D camera in the environment to collect data. I hope the tile camera which could help speed up the rendering time can move and support more advanced lighting effects. This issue.

3. How would you rate the support provided through GitHub issues and discussions? (1-10,1 ='Support?!, 10='Stop reading my mind!')

Ans: 8

4. Would you like to see more open-source projects that allow for your contributions?

Ans: I am wondering if there are research plans available that allow laboratories using IsaacLab to submit research proposals.

5. How easy was it to get started with the framework? (1-10, 1='I spent months before understanding basics', 10-'I understood everything in an hour'). Were there any particular pain points during installation or initial setup?

Ans: 7 It worked quite well on the physical machine. However, when I tried to install it on the school server using Singularity, it took me a while to figure out that I needed to set MESA_GL_VERSION_OVERRIDE=4.6 and use CUDA_VISIBLE_DEVICES=1 with the --headless option. I believe the issue is specific to the school's server though.

6. Have the examples and tutorials provided been helpful for understanding key concepts? What additional examples or tutorials would you like to see?

Ans: I hope there are more tutorials available on configuring and using RGB-D cameras. For example, guidelines on how to move the camera from one set of XYZ coordinates to another through teleporting and ways to speed up simulations with the RGB-D camera enabled. Currently, I feel that rendering significantly slows down the simulation.

7. Have you encountered any performance issues or bottlenecks? If so, how could these be improved?

Ans: This is probably not your issue. I followed the guidelines in Section 4.1 of this paper to build the environment. In their case, they used Isaac Gym and achieved speeds of up to 1000 FPS with 256 parallel environments. However, when using IsaacLab, my speed is around 30 FPS with 8 parallel environments. I'm still trying to determine whether this is a bottleneck or due to a bug in my setup. In addition, without the help of the tile camera, I feel like the fps won't increase too much when adding environment.

8. How does the framework compare to other tools or libraries you've used for similar purposes? What other libraries have you used for your RL needs?

Ans: I used Webots for simulation in the past, and its environment is visually appealing. However, I found it less user-friendly for programming and my research needs. Additionally, the speed slows down significantly when I add more cameras (6-10) or increase the camera resolution.

8. Anything else you can think of?

Ans: Nope :)

[jsmith-bdai]

Thank you for your feedback!

[cidxb]

Has the organization and structure of code been easy to understand? (1-10, 1='What structure?', 10='The structure was very intuitive and easy to grasp')
Ans: 7

Are there any features or integrations you would like to see prioritized in upcoming releases?
Ans: Maybe Universal teleoperation interface through camera for robotic?

How would you rate the support provided through GitHub issues and discussions? (1-10,1 ='Support?!, 10='Stop reading my mind!')
Ans: 8

Would you like to see more open-source projects that allow for your contributions?
Ans: YES

How easy was it to get started with the framework? (1-10, 1='I spent months before understanding basics', 10-'I understood everything in an hour'). Were there any particular pain points during installation or initial setup?
Ans: 8

Have the examples and tutorials provided been helpful for understanding key concepts? What additional examples or tutorials would you like to see?
Ans: Helpful, but I would like to see more project implementations example, which i consider this much easier and helpful to learn the usages throughexample.

Have you encountered any performance issues or bottlenecks? If so, how could these be improved?
Ans: not yet

How does the framework compare to other tools or libraries you've used for similar purposes? What other libraries have you used for your RL needs?
Ans: I used to use SB3 for my needs. But I am not yet really dive into IsaacLab yet, hard to tell.

Anything else you can think of?
Ans: Still very new to the IsaacLab. The Manager-Based Environments style for building up RL environment, is brilliant, which I also thought of by myself in the past. But seeing the actual implements, it expose the shortcoming: It is a "modular implementations of tasks by decomposing the task into individual components ", so it costs to get start with this kind of RL env building style, since you need to have a grasp of knowing each components and what they contain firstly. This kind of building style might be a obstacle for beginner and who wants to build up a environment rapidly and under constant alteration. Of course there is always direct-style. I just wondering, if there a better way presenting the Manager-Based Environments.

[MoreTore]

I would like more documentation of how to accurately model physical actuators. For example, the LSTM Network Actuator and the MLP Network Actuator provide a way for engineers to add data driven models of physical actuators. However, there is no instruction on how these models are created except for this paper here https://arxiv.org/pdf/1901.08652 which refers to this repo https://github.com/junja94/anymal_science_robotics_supplementary.

For more basic actuators like wheels, it would be great if there was a guide on how one can accurately model these as well. In my case, i have a hoverboard wheel controlled by a odrive BLDC controller. How do the parameters in ActuatorBaseCfg relate to the real characteristics of the physical wheel? A better description of velocity_limit, stiffness, damping, armature, friction, saturation_effort, and how to measure them in the physical motor would be nice to have.

Maybe someone at NVIDIA can create a omniverse LLM based on the documentation, codebase, and forums because it feels like the docs are fragmented and you need to know where to look to find the answer you need.

Besides that, I am very happy with isaac lab and isaac sim. I was able to import my robot design from fusion 360, setup the joints and train a balancing robot with command-able height, pitch, yaw, and velocity within a few days. Very good experience in that respect.

[SakethTecholution]

This is true , even I felt the same

[SakethTecholution]

Has the organization and structure of code been easy to understand? (1-10, 1='What structure?', 10='The structure was very intuitive and easy to grasp') (7)
The structure is intuitive but since its so modular, as a beginner it took me som much time to understand things

Are there any features or integrations you would like to see prioritized in upcoming releases?

• I would sasy most importantly tutorials need to improve, especially if we see tutorials for isaac sim , they have clearly provided how to import new robot , just like that how to simulate a new robot with usd's need to there in Isaac lab tutorials as well . Because , I felt too tough to grasp until i see lot of example code . ( WE NEED MORE TUTORIALS - VIDEO DEMONSTRATIONS )

How would you rate the support provided through GitHub issues and discussions? (1-10,1 ='Support?!, 10='Stop reading my mind!')
(6) - most of my issues which kept in github , they took very long time to reply. Even before that we had resolved the issue ..

Would you like to see more open-source projects that allow for your contributions
Yes, especially in the field of imitation learning . using recently most used policies like ACT & DIFFUSION need to be added and the current robomiic doesnt works even after so many demonstartions . Need to properlyt check on this and tutorials need to be provided

How easy was it to get started with the framework? (1-10, 1='I spent months before understanding basics', 10-'I understood everything in an hour'). Were there any particular pain points during installation or initial setup?
It took me couple of weeks to grasp because of no such good tutorials available online neither in IsaacLab

Have the examples and tutorials provided been helpful for understanding key concepts? What additional examples or tutorials would you like to see?
More examples are required with video demonstrations

Have you encountered any performance issues or bottlenecks? If so, how could these be improved?
I got lot of GPU issues while i am using Isaaclab

How does the framework compare to other tools or libraries you've used for similar purposes? What other libraries have you used for your RL needs?
I felt mojoco is useful since it has good documentatiion .. didnt tried much but lof of people in this field uses that and I found lot of opensourced code and community there

Anything else you can think of?
We should have a discord community for bettter community discussions. Community is the most important part to make isaacLab used by more people and there need to be some people from nvidea team to support new starters into this field

Tutorials were required on Using Isaac sim tools liek omnigraphs and much more( Felt Isaac Sim tutorials are good and well documented , but i want to use them along with Isaac lab , but there is no documentation on it )

[rj-maniro]

+100 to @SakethTecholution's suggestion to start an IsaacLabs Discord server. This Github discussions is insufficient for creating a vibrant IsaacLabs community. @jsmith-bdai - would you and your team be able to start one?

Also, it looks like the page with example / demo scripts has been taken down. That was very helpful for getting started. Can we have more examples and tutorials please?

[MoreTore]

There already is a discord for Isaac sim https://discord.gg/Yy6zmEWP

[rj-maniro]

Cool. Should we start an Isaac Labs channel in there then?