Updates docs for newton beta 2 (#4238)

# Description

Updates documentation for newton beta 2:

- Installation instructions
- Visualizers instructions
- Multi-backends - folder restructuring and mechanism to switch between
backends
- Quaternion convention change - switching to XYZW
- Warp dependency changes - data will be returned in Warp and users need
to call wp.to_torch to convert to torch


## Type of change

- Documentation update

## Checklist

- [x] I have read and understood the [contribution
guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html)
- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [ ] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it

For example,
- [x] I have done this task
- [ ] I have not done this task
-->

---------

Signed-off-by: James Tigue <[email protected]>
Signed-off-by: ooctipus <[email protected]>
Signed-off-by: Kelly Guo <[email protected]>
Signed-off-by: Brian McCann <[email protected]>
Signed-off-by: Kelly Guo <[email protected]>
Signed-off-by: Fan Dongxuan <[email protected]>
Signed-off-by: renezurbruegg <[email protected]>
Signed-off-by: Mayank Mittal <[email protected]>
Signed-off-by: Pascal Roth <[email protected]>
Signed-off-by: matthewtrepte <[email protected]>
Co-authored-by: Antoine RICHARD <[email protected]>
Co-authored-by: James Tigue <[email protected]>
Co-authored-by: Eva M. <[email protected]>
Co-authored-by: James Smith <[email protected]>
Co-authored-by: ooctipus <[email protected]>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Pascal Roth <[email protected]>
Co-authored-by: Toni-SM <[email protected]>
Co-authored-by: Yanzi Zhu <[email protected]>
Co-authored-by: Greg Attra <[email protected]>
Co-authored-by: rwiltz <[email protected]>
Co-authored-by: Brian McCann <[email protected]>
Co-authored-by: Krishna Lakhi <[email protected]>
Co-authored-by: Kyle Morgenstein <[email protected]>
Co-authored-by: Mayank Mittal <[email protected]>
Co-authored-by: Fan Dongxuan <[email protected]>
Co-authored-by: Ashwin Varghese Kuruttukulam <[email protected]>
Co-authored-by: renezurbruegg <[email protected]>
Co-authored-by: Pascal Roth <[email protected]>
Co-authored-by: Mayank Mittal <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: zrene <[email protected]>
Co-authored-by: yami007007-weihuaz <[email protected]>
Co-authored-by: matthewtrepte <[email protected]>

Author: 22 people

docs/source/_static/visualizers/newton_viz.jpg

@@ -38,8 +38,9 @@ until the framework has reached an official release. We appreciate your understa
   :titlesonly:
 
   installation
+  isaaclab_3-0
   training-environments
-  newton-visualizer
+  visualization
   limitations-and-known-bugs
   solver-transitioning
   sim-to-sim

docs/source/_static/visualizers/ov_viz.jpg

@@ -3,13 +3,13 @@ Installation
 
 Installing the Newton physics integration branch requires three things:
 
-1) Isaac sim 5.0
-2) The ``feature/newton`` branch of Isaac Lab
-3) Ubuntu 22.04 or 24.04 (Windows will be supported soon)
+1) The ``feature/newton`` branch of Isaac Lab
+2) Ubuntu 22.04 or 24.04 (Windows will be supported soon)
+3) [Optional] Isaac sim 5.1 (Isaac Sim is not required if the Omniverse visualizer is not used)
 
 To begin, verify the version of Isaac Sim by checking the title of the window created when launching the simulation app.  Alternatively, you can
 find more explicit version information under the ``Help -> About`` menu within the app.
-If your version is less than 5.0, you must first `update or reinstall Isaac Sim <https://docs.isaacsim.omniverse.nvidia.com/latest/installation/quick-install.html>`_ before
+If your version is less than 5.1, you must first `update or reinstall Isaac Sim <https://docs.isaacsim.omniverse.nvidia.com/latest/installation/quick-install.html>`_ before
 you can proceed further.
 
 Next, navigate to the root directory of your local copy of the Isaac Lab repository and open a terminal.
@@ -20,14 +20,18 @@ Make sure we are on the ``feature/newton`` branch by running the following comma
 
     git checkout feature/newton
 
-Below, we provide instructions for installing Isaac Sim through pip or binary.
+Below, we provide instructions for installing Isaac Sim through pip.
 
 
 Pip Installation
 ----------------
 
 We recommend using conda for managing your python environments. Conda can be downloaded and installed from `here <https://docs.conda.io/en/latest/miniconda.html>`_.
 
+If you previously already have a virtual environment for Isaac Lab, please ensure to start from a fresh environment to avoid any dependency conflicts.
+If you have installed earlier versions of mujoco, mujoco-warp, or newton packages through pip, we recommend first
+cleaning your pip cache with ``pip cache purge`` to remove any cache of earlier versions that may be conflicting with the latest.
+
 Create a new conda environment:
 
 .. code-block:: bash
@@ -46,35 +50,11 @@ Install the correct version of torch and torchvision:
 
     pip install -U torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128
 
-Install Isaac Sim 5.0:
-
-.. code-block:: bash
-
-    pip install "isaacsim[all,extscache]==5.0.0" --extra-index-url https://pypi.nvidia.com
-
-Install Isaac Lab extensions and dependencies:
-
-.. code-block:: bash
-
-    ./isaaclab.sh -i
-
-
-Binary Installation
--------------------
-
-Follow the Isaac Sim `documentation <https://docs.isaacsim.omniverse.nvidia.com/latest/installation/install_workstation.html>`_ to install Isaac Sim 5.0 binaries.
-
-Enter the Isaac Lab directory:
-
-.. code-block:: bash
-
-    cd IsaacLab
-
-Add a symbolic link to the Isaac Sim installation:
+[Optional] Install Isaac Sim 5.1:
 
 .. code-block:: bash
 
-    ln -s path_to_isaac_sim _isaac_sim
+    pip install "isaacsim[all,extscache]==5.1.0" --extra-index-url https://pypi.nvidia.com
 
 Install Isaac Lab extensions and dependencies:
 
@@ -91,3 +71,8 @@ To verify that the installation was successful, run the following command from t
 .. code-block:: bash
 
     ./isaaclab.sh -p scripts/environments/zero_agent.py --task Isaac-Cartpole-Direct-v0 --num_envs 128
+
+
+Note that since Newton requires a more recent version of Warp than Isaac Sim 5.1, there may be some incompatibility issues
+that could result in errors such as ``ModuleNotFoundError: No module named 'warp.sim'``. These are ok to ignore and should not
+impact usability.

docs/source/_static/visualizers/rerun_viz.jpg

@@ -0,0 +1,49 @@
+IsaacLab 3.0
+============
+
+IsaacLab 3.0 is the upcoming release of IsaacLab, which will be compatible with Isaac Sim 6.0, and at the same time will support the new Newton physics engine.
+This will allow users to train policies on the Newton physics engine, or PhysX. To accommodate this major code refactoring are required. In this section, we
+will go over some of the changes, how that will affect IsaacLab 2.X users, and how to migrate to IsaacLab 3.0. The current branch of ``feature/newton`` gives
+a glance of what is to come. While the changes to the internal code structure are significant, the changes to the user API are minimal.
+
+
+Let's start with the biggest change: enabling the use of multiple physics engines. We refactored our code so that we can not only support PhysX and Newton, but
+any other physics engine, enabling users to bring their own physics engine to IsaacLab if they desire. To enable this, we introduce a base implementations of
+our ``simulation interfaces``, :class:`~isaaclab.assets.articulation.Articulation` or :class:`~isaaclab.sensors.ContactSensor` for instance. These provide a
+set of abstract methods that all physics engines must implement. In turn this allows all of the default IsaacLab environments to work with any physics engine.
+This also allows us to ensure that IsaacLab 3.0 is backwards compatible with IsaacLab 2.X. For engine specific calls, users could get the underlying view of
+the physics engine and call the engine specific APIs directly.
+
+However, as we are refactoring the code, we are also looking at ways to limit the overhead of IsaacLab's. In an effort to minimize the overhead, we are moving
+all our low level code away from torch, and instead will rely heavily on warp. This will allow us to write low level code that is more efficient, and also
+to take advantage of the cuda-graphing. However, this means that the ``data classes`` such as :class:`~isaaclab.assets.articulation.ArticulationData` or
+:class:`~isaaclab.sensors.ContactSensorData` will only return warp arrays. Users will hence have to call ``wp.to_torch`` to convert them to torch tensors if they desire.
+Our setters/writers will support both warp arrays and torch tensors, and will use the most optimal strategy to update the warp arrays under the hood. This minimizes the
+amount of changes required for users to migrate to IsaacLab 3.0.
+
+Another new feature of the writers and setters is the ability to provide them with masks and complete data (as opposed to indices and partial data in IsaacLab 2.X).
+Note that this feature will be available along with the ability to provide indices and partial data, and that the default behavior will still be to provide indices and partial data.
+However, if using warp, users will have to provide masks and complete data. In general we encourage users to move to adopt this new feature as, if done well, it will
+reduce on the fly memory allocations, and should result in better performance.
+
+On the optimization front, we decided to change quaternion conventions. Originally, IsaacLab and Isaac Sim both adopted the ``wxyz`` convention. However, we were doing a located
+of conversion to and from ``xyzw`` in our setters/writers as PhysX uses the ``xyzw`` convention. Since both Newton and Warp, also use the ``xyzw`` convention, we decided to change
+our default convention to ``xyzw``. This means that all our APIs will now return quaternions in the ``xyzw`` convention. This is likely a breaking change for all the custom
+mdps that are not using our :mod:`~isaaclab.utils.math` module. While this change is substantial, it should make things more consistent for when users are using the simulation
+views directly, and will remove needless conversions.
+
+Finally, alongside the new isaaclab_newton extension, we are also introducing new isaaclab_experimental and isaaclab_task_experimental extensions. These extensions will allow
+quickly bring to main isaaclab new features while leaving them the time they need to mature before being fully integrated into the core isaaclab extensions. In this release,
+we are introducing cuda-graphing support for direct rl tasks. This allows to drastically reduce IsaacLab's overhead making training faster. Try them out and let us know what you think!
+
+.. code-block:: bash
+
+    ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Cartpole-Direct-Warp-v0 --num_envs 4096 --headless
+
+.. code-block:: bash
+
+    ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Ant-Direct-Warp-v0 --num_envs 4096 --headless
+
+.. code-block:: bash
+
+    ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Humanoid-Direct-Warp-v0 --num_envs 4096 --headless

docs/source/experimental-features/newton-physics-integration/index.rst

@@ -48,7 +48,6 @@ Here is a non-exhaustive list of capabilities currently supported in the Newton
             * A1
             * Go1
             * Go2
-            * Spot
             * Unitree G1
             * Unitree H1
         * Manipulation reach

docs/source/experimental-features/newton-physics-integration/installation.rst

@@ -40,7 +40,7 @@ Train the teacher policy for the G1 velocity task using the Newton backend. The
 
 .. code-block:: bash
 
-   ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task=Isaac-Velocity-Flat-G1-v1 --num_envs=4096 --headless
+   ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task=Isaac-Velocity-Flat-G1-v1 --num_envs=4096
 
 The teacher policy includes privileged observations (e.g., root linear velocity) defined in ``PolicyCfg(ObsGroup)``.
 
@@ -59,7 +59,7 @@ Run the student distillation task ``Velocity-G1-Distillation-v1`` using ``--load
 
 .. code-block:: bash
 
-   ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task=Velocity-G1-Distillation-v1 --num_envs=4096 --headless --load_run 2025-08-13_23-53-28 --checkpoint model_1499.pt
+   ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task=Velocity-G1-Distillation-v1 --num_envs=4096 --load_run 2025-08-13_23-53-28 --checkpoint model_1499.pt
 
 .. note::
 
@@ -74,7 +74,7 @@ Use ``--load_run`` and ``--checkpoint`` to initialize from the distilled policy.
 
 .. code-block:: bash
 
-   ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task=Velocity-G1-Student-Finetune-v1 --num_envs=4096 --headless --load_run 2025-08-20_16-06-52_distillation --checkpoint model_1499.pt
+   ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task=Velocity-G1-Student-Finetune-v1 --num_envs=4096 --load_run 2025-08-20_16-06-52_distillation --checkpoint model_1499.pt
 
 This starts from the distilled student policy and improves it further with RL training.
 
@@ -86,7 +86,7 @@ You can replay the student policy via:
 
 .. code-block:: bash
 
-   ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py --task=Velocity-G1-Student-Finetune-v1 --num_envs=32
+   ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py --task=Velocity-G1-Student-Finetune-v1 --num_envs=32 --visualizer newton
 
 
 This exports the policy as ``.pt`` and ``.onnx`` files in the run's export directory, ready for real robot deployment.

docs/source/experimental-features/newton-physics-integration/isaaclab_3-0.rst

@@ -54,7 +54,8 @@ To run a PhysX-trained policy with the Newton backend, use this command template
        --task=<TASK_ID> \
        --num_envs=32 \
        --checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
-       --policy_transfer_file <PATH_TO_MAPPING_YAML>
+       --policy_transfer_file <PATH_TO_MAPPING_YAML> \
+       --visualizer newton
 
 Here are examples for different robots:
 
@@ -66,8 +67,8 @@ Here are examples for different robots:
        --task=Isaac-Velocity-Flat-G1-v0 \
        --num_envs=32 \
        --checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
-       --policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_g1.yaml
-
+       --policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_g1.yaml \
+       --visualizer newton
 
 2. Unitree H1
 
@@ -78,7 +79,8 @@ Here are examples for different robots:
        --task=Isaac-Velocity-Flat-H1-v0 \
        --num_envs=32 \
        --checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
-       --policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_h1.yaml
+       --policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_h1.yaml \
+       --visualizer newton
 
 
 3. Unitree Go2
@@ -89,7 +91,8 @@ Here are examples for different robots:
        --task=Isaac-Velocity-Flat-Go2-v0 \
        --num_envs=32 \
        --checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
-       --policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_go2.yaml
+       --policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_go2.yaml \
+       --visualizer newton
 
 
 4. ANYmal-D
@@ -101,7 +104,8 @@ Here are examples for different robots:
        --task=Isaac-Velocity-Flat-Anymal-D-v0 \
        --num_envs=32 \
        --checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
-       --policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_anymal_d.yaml
+       --policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_anymal_d.yaml \
+       --visualizer newton
 
 Note that to run this, you need to checkout the Newton-based branch of IsaacLab such as ``feature/newton``.