Error When modifiying a mesh with warp using the USDRT scene graph example

[SantiDiazC]

Description

Hi,
I am trying to modify a mesh inside Isaacsim so It may be possible to control deformation of active elements like soft actuators or soft robots, so for this I try to run, on the python script editor, the basic example provided here (using warp to modify a Torus mesh in the USDRT scene graph): https://docs.omniverse.nvidia.com/kit/docs/usdrt/latest/docs/whatsinfabric.html

The problem is when run it in the python script editor the Torus in the example disappeared. I tried in an older version of USD composer, more specifically v 2023.2.5 beta and it runs well. I don't know if there is some compatibility Issue realted.

The code I use in the editor is like this:

import omni.usd
from usdrt import Usd, Sdf, UsdGeom, Rt, Gf, Vt
import warp as wp

stage_id = omni.usd.get_context().get_stage_id()

stage = Usd.Stage.Attach(stage_id)

prim_path = "/World/Torus"

time = 10

from usdrt import Usd, Sdf, Gf, Vt, Rt

import warp as wp
wp.init()

@wp.kernel
def deform(positions: wp.array(dtype=wp.vec3), t: float):
    tid = wp.tid()

    x = positions[tid]
    offset = -wp.sin(x[0])
    scale = wp.sin(t)*10.0

    x = x + wp.vec3(0.0, offset*scale, 0.0)

    positions[tid] = x

def deform_mesh_with_warp(stage_id, path, time):
    """Use Warp to deform a Mesh prim
    """
    if path is None:
        return "Nothing selected"

    stage = Usd.Stage.Attach(stage_id)
    prim = stage.GetPrimAtPath(Sdf.Path(path))
    if not prim:
        return f"Prim at path {path} is not in Fabric"

    if not prim.HasAttribute("points"):
        return f"Prim at path {path} does not have points attribute"

    # Tell OmniHydra to render points from Fabric
    if not prim.HasAttribute("Deformable"):
        prim.CreateAttribute("Deformable", Sdf.ValueTypeNames.PrimTypeTag, True)

    points = prim.GetAttribute("points")
    pointsarray = points.Get()
    warparray = wp.array(pointsarray, dtype=wp.vec3, device="cuda")

    wp.launch(
        kernel=deform,
        dim=len(pointsarray),
        inputs=[warparray, time],
        device="cuda"
    )

    points.Set(Vt.Vec3fArray(warparray.numpy()))

    return f"Deformed points on prim {path}"

deform_mesh_with_warp(stage_id, prim_path, time)

If there is a way to solve it please let me know or if you know a better way to achieve what I want inside Isaac sim and later Isaaclab I will appreciate a lot, Thank you!

Isaac Sim version

5.0.0

Operating System (OS)

Ubuntu 22.04

GPU Name

RTX 3080

GPU Driver and CUDA versions

Driver 570.195.03 and CUDA 12.8

Logs

When I run the code it is cached and then the error appears in the terminal:

Module __main__ f4dbe7d load on device 'cuda:0' took 5.05 ms  (cached)
2025-10-28T06:59:12Z [106,265ms] [Warning] [omni.hydra] Mesh '/World/Torus' update topology/point without updating normal, fallback to smooth normal.
2025-10-28T06:59:12Z [106,287ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,287ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,287ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,287ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,287ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,287ms] [Warning] [omni.hydra] Rendering failed.
2025-10-28T06:59:12Z [106,287ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:12Z [106,292ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,292ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,292ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,292ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,292ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,292ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:12Z [106,326ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,326ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,326ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,326ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,326ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,326ms] [Warning] [omni.hydra] Rendering failed.
2025-10-28T06:59:12Z [106,326ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:12Z [106,341ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,341ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,341ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,341ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,341ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,341ms] [Warning] [omni.hydra] Rendering failed.
2025-10-28T06:59:12Z [106,341ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:12Z [106,365ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,365ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,365ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,365ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,365ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,365ms] [Warning] [omni.hydra] Rendering failed.
2025-10-28T06:59:12Z [106,366ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:12Z [106,387ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,387ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,387ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,387ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,387ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,388ms] [Warning] [omni.hydra] Rendering failed.
2025-10-28T06:59:12Z [106,388ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:12Z [106,395ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,395ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,395ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,395ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,395ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,395ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:12Z [106,421ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,421ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,421ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,421ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,421ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,421ms] [Warning] [omni.hydra] Rendering failed.
2025-10-28T06:59:12Z [106,421ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:12Z [106,429ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,429ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,429ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,429ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,429ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,429ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:12Z [106,463ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,463ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,463ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,463ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,463ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,463ms] [Warning] [omni.hydra] Rendering failed.
2025-10-28T06:59:12Z [106,463ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:12Z [106,478ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,478ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,478ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,478ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,478ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,478ms] [Warning] [omni.hydra] Rendering failed.
2025-10-28T06:59:12Z [106,478ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:12Z [106,515ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,515ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,515ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,515ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,515ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,515ms] [Warning] [omni.hydra] Rendering failed.
2025-10-28T06:59:12Z [106,515ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:12Z [106,550ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,550ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,550ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,550ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,550ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,550ms] [Warning] [omni.hydra] Rendering failed.
2025-10-28T06:59:12Z [106,550ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:12Z [106,556ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:12Z [106,556ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:12Z [106,556ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:12Z [106,556ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:12Z [106,556ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:12Z [106,556ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:13Z [106,579ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:13Z [106,579ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:13Z [106,579ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:13Z [106,579ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:13Z [106,579ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:13Z [106,579ms] [Warning] [omni.hydra] Rendering failed.
2025-10-28T06:59:13Z [106,579ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:13Z [106,593ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:13Z [106,593ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:13Z [106,593ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:13Z [106,593ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:13Z [106,593ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:13Z [106,593ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:13Z [106,614ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:13Z [106,614ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:13Z [106,614ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:13Z [106,614ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:13Z [106,614ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:13Z [106,614ms] [Warning] [omni.hydra] Rendering failed.
2025-10-28T06:59:13Z [106,614ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:13Z [106,623ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:13Z [106,623ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:13Z [106,623ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:13Z [106,623ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:13Z [106,623ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:13Z [106,623ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:13Z [106,648ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:13Z [106,648ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:13Z [106,648ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:13Z [106,648ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:13Z [106,648ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:13Z [106,648ms] [Warning] [omni.hydra] Rendering failed.
2025-10-28T06:59:13Z [106,648ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:13Z [106,656ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:13Z [106,656ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:13Z [106,656ms] [Error] [gpu.foundation.plugin] Cannot create cuda external memory for resource!
2025-10-28T06:59:13Z [106,656ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2025-10-28T06:59:13Z [106,656ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to allocate 1280x720 LdrColor resource for device mask 0x1
2025-10-28T06:59:13Z [106,656ms] [Error] [omni.usd] HydraEngine::render failed to end the compute graph: error code 6
2025-10-28T06:59:13Z [106,690ms] [Error] [carb.cudainterop.plugin] CUDA error 700: cudaErrorIllegalAddress - an illegal memory access was encountered)
2025-10-28T06:59:13Z [106,690ms] [Error] [carb.cudainterop.plugin] Failed to import external memory in CUDA
2025-10-28T06:59:13Z [106,690ms] [Error] [gpu.foundation

When I click Run on the GUI to run the simulation this error appears in the terminal:

2025-10-28T07:01:07Z [0ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] A crash has occurred.  If a debugger should be attached, please set the '/crashreporter/debuggerAttachTimeoutMs' setting to a timeout in milliseconds.  This can be used to allow the crash reporter to wait for up to that long for a debugger to attach before processing or sending the crash report.
2025-10-28T07:01:38Z [30,209ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:02:08Z [60,392ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:02:38Z [90,573ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:03:08Z [120,757ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:03:38Z [150,935ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:04:08Z [181,116ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:04:39Z [211,297ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:05:09Z [241,481ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:05:39Z [271,661ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:06:09Z [301,848ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:06:39Z [332,035ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:07:10Z [362,221ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:07:40Z [392,405ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:08:10Z [422,590ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:08:40Z [452,774ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:09:10Z [482,956ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:09:40Z [513,137ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:10:11Z [543,322ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:10:41Z [573,503ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:11:11Z [603,689ms] [Warning] [carb.crashreporter-breakpad.plugin]    ... waiting for PID 2377617 to exit ...
2025-10-28T07:11:11Z [603,700ms] [Warning] [carb.crashreporter-breakpad.plugin]    Giving up on PID 2377617
2025-10-28T07:11:11Z [603,706ms] [Error] [carb.crashreporter-breakpad.plugin] [crash] Wrote dump file '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.dmp'
2025-10-28T07:11:11Z [603,710ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] Uploading minidump: file:'/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.dmp' svr:'https://services.nvidia.com/submit'
2025-10-28T07:11:11Z [603,715ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] Crash metadata for upload:
2025-10-28T07:11:11Z [603,719ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'BuildGitlabJobID' = '182268558'
2025-10-28T07:11:11Z [603,724ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'CarbSdkVersion' = '206.4+release.9396.3c6f8191.gl'
2025-10-28T07:11:11Z [603,738ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'CrashTime' = 'Tue Oct 28 07:01:07 2025 GMT'
2025-10-28T07:11:11Z [603,751ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'DriverShaderCacheWrapper' = 'disabled'
2025-10-28T07:11:11Z [603,758ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'DumpId' = '31a5493b-dd1f-4322-936ef490-c620a8c6'
2025-10-28T07:11:11Z [603,760ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'LastUploadStatus' = ''
2025-10-28T07:11:11Z [603,761ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'ProductName' = 'OmniverseKit'
2025-10-28T07:11:11Z [603,763ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'PythonTracebackStatus' = ''py-spy' successfully wrote info to '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.py.txt''
2025-10-28T07:11:11Z [603,765ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'RetryCount' = '0'
2025-10-28T07:11:11Z [603,767ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'StartupTime' = '1761634646'
2025-10-28T07:11:11Z [603,768ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'UploadSuccessful' = '0'
2025-10-28T07:11:11Z [603,770ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'UptimeSeconds' = '221'
2025-10-28T07:11:11Z [603,772ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'UserStory' = ''
2025-10-28T07:11:11Z [603,774ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'UserStoryStatus' = ''crashreport.gui' timed out'
2025-10-28T07:11:11Z [603,775ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'Version' = '107.3.1+production.206797.8131b85d.gl'
2025-10-28T07:11:11Z [603,777ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'aftermath_enabled' = '1'
2025-10-28T07:11:11Z [603,779ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'aftermath_flags' = '3'
2025-10-28T07:11:11Z [603,784ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'aftermath_status' = 'auto-enabled'
2025-10-28T07:11:11Z [603,786ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'aftermath_version' = '2024.1'
2025-10-28T07:11:11Z [603,788ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'appName' = 'Isaac-Sim Full'
2025-10-28T07:11:11Z [603,790ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'appState' = 'started'
2025-10-28T07:11:11Z [603,793ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'appVersion' = '5.0.0'
2025-10-28T07:11:11Z [603,795ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'autoloadExts' = ''
2025-10-28T07:11:11Z [603,798ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'buildBranch' = 'production'
2025-10-28T07:11:11Z [603,800ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'buildCi' = 'gl'
2025-10-28T07:11:11Z [603,803ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'buildConfig' = 'release'
2025-10-28T07:11:11Z [603,805ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'buildGitlabJobName' = 'kit-build-release-linux-x86_64'
2025-10-28T07:11:11Z [603,808ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'buildGitlabJobStage' = 'kit-build'
2025-10-28T07:11:11Z [603,810ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'buildGitlabPipelineID' = '30713850'
2025-10-28T07:11:11Z [603,817ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'buildHash' = '8131b85d'
2025-10-28T07:11:11Z [603,819ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'buildMajor' = '107'
2025-10-28T07:11:11Z [603,822ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'buildMinor' = '3'
2025-10-28T07:11:11Z [603,824ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'buildNumber' = '206797'
2025-10-28T07:11:11Z [603,827ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'buildPatch' = '1'
2025-10-28T07:11:11Z [603,829ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'buildVersion' = '107.3.1'
2025-10-28T07:11:11Z [603,832ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'carboniteFrameworkVersion' = '206.4+release.9396.3c6f8191.gl'
2025-10-28T07:11:11Z [603,835ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'carboniteSdkVersion' = '206.4+release.9396.3c6f8191.gl'
2025-10-28T07:11:11Z [603,838ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'commandLine' = '/home/<username>/IsaacSim/_build/linux-x86_64/release/kit/kit /home/<username>/IsaacSim/_build/linux-x86_64/release/apps/isaacsim.exp.full.kit'
2025-10-28T07:11:11Z [603,842ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'cpuCoreLimited' = '20'
2025-10-28T07:11:11Z [603,845ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'cpuId' = 'Intel64 Family 6 Model 151 Stepping 2'
2025-10-28T07:11:11Z [603,849ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'cpuLogicalCoresBareMetal' = '24'
2025-10-28T07:11:11Z [603,852ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'cpuName' = '12th Gen Intel(R) Core(TM) i7-12700K'
2025-10-28T07:11:11Z [603,855ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'cpuPhysicalCoresBareMetal' = '12'
2025-10-28T07:11:11Z [603,859ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'cpuUsageQuota' = '-1.000000'
2025-10-28T07:11:11Z [603,862ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'cpuVendor' = 'GenuineIntel'
2025-10-28T07:11:11Z [603,866ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'desktopOrigin' = '(0, 0)'
2025-10-28T07:11:11Z [603,869ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'desktopSize' = '6000x3840'
2025-10-28T07:11:11Z [603,872ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'displayCount' = '2'
2025-10-28T07:11:11Z [603,876ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'displayRes_0' = '3840x2160x32bit@60Hz'
2025-10-28T07:11:11Z [603,879ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'displayRes_1' = '1920x1080x32bit@60Hz'
2025-10-28T07:11:11Z [603,883ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'email' = ''
2025-10-28T07:11:11Z [603,887ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'environmentName' = 'Individual'
2025-10-28T07:11:11Z [603,891ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'externalBuild' = '1'
2025-10-28T07:11:11Z [603,895ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'extraExts' = 'isaacsim.ros2.bridge-4.9.3'
2025-10-28T07:11:11Z [603,899ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'gpuDriver_0' = '570.195'
2025-10-28T07:11:11Z [603,903ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'gpuVRAM_0' = '10995367936'
2025-10-28T07:11:11Z [603,907ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'gpu_0' = 'NVIDIA GeForce RTX 3080'
2025-10-28T07:11:11Z [603,912ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'kitRendererDriverVersion' = '570.195'
2025-10-28T07:11:11Z [603,916ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'lastCommand' = 'ToolbarPlayButtonClicked'
2025-10-28T07:11:11Z [603,920ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'lastCommands' = 'SetLightingMenuModeCommand(lighting_mode=stage,usd_context_name=),CreateMeshPrimWithDefaultXform(prim_type=Torus,above_ground=True),ToolbarPlayButtonClicked'
2025-10-28T07:11:11Z [603,924ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'lib_isaacSim_buildBranch' = 'main'
2025-10-28T07:11:11Z [603,928ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'lib_isaacSim_buildDate' = 'Fri Aug 8 12:39:04 2025 -0500'
2025-10-28T07:11:11Z [603,932ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'lib_isaacSim_buildHash' = '1791e14'
2025-10-28T07:11:11Z [603,936ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'lib_isaacSim_buildRepo' = 'https://github.com/isaac-sim/IsaacSim.git'
2025-10-28T07:11:11Z [603,945ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'lib_isaacSim_buildVersion' = '5.0.0-rc.45'
2025-10-28T07:11:11Z [603,958ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'lib_physx_buildBranch' = 'HEAD'
2025-10-28T07:11:11Z [603,965ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'lib_physx_buildDate' = 'Jul-23-2025'
2025-10-28T07:11:11Z [603,967ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'lib_physx_buildHash' = '214216c'
2025-10-28T07:11:11Z [603,969ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'lib_physx_buildRepo' = 'gitlab-master.nvidia.com/omniverse/physics'
2025-10-28T07:11:11Z [603,970ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'lib_physx_buildVersion' = '107.3.18'
2025-10-28T07:11:11Z [603,972ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'memoryStats' = '(avail/total) RAM: 43.3496/62.5391GB, Swap: 2/2GB, VM: 1.71799e+10/1.71799e+10GB'
2025-10-28T07:11:11Z [603,974ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'nvidia-smi' = '
==============NVSMI LOG==============

Timestamp                                 : Tue Oct 28 15:57:26 2025
Driver Version                            : 570.195.03
CUDA Version                              : 12.8

Attached GPUs                             : 1
GPU 00000000:01:00.0
    Product Name                          : NVIDIA GeForce RTX 3080
    Product Brand                         : GeForce
    Product Architecture                  : Ampere
    Display Mode                          : Enabled
    Display Active                        : Enabled
    Persistence Mode                      : Disabled
    Addressing Mode                       : None
    MIG Mode
        Current                           : N/A
        Pending                           : N/A
    Accounting Mode                       : Disabled
    Accounting Mode Buffer Size           : 4000
    Driver Model
        Current                           : N/A
        Pending                           : N/A
    Serial Number                         : N/A
    GPU UUID                              : GPU-dbf1aa89-d3ee-8795-21f6-b2af45adb2cc
    Minor Number                          : 0
    VBIOS Version                         : 94.02.26.88.2A
    MultiGPU Board                        : No
    Board ID                              : 0x100
    Board Part Number                     : N/A
    GPU Part Number                       : 2206-200-A1
    FRU Part Number                       : N/A
    Platform Info
        Chassis Serial Number             : N/A
        Slot Number                       : N/A
        Tray Index                        : N/A
        Host ID                           : N/A
        Peer Type                         : N/A
        Module Id                         : 1
        GPU Fabric GUID                   : N/A
    Inforom Version
        Image Version                     : G001.0000.03.03
        OEM Object                        : 2.0
        ECC Object                        : N/A
        Power Management Object           : N/A
    Inforom BBX Object Flush
        Latest Timestamp                  : N/A
        Latest Duration                   : N/A
    GPU Operation Mode
        Current                           : N/A
        Pending                           : N/A
    GPU C2C Mode                          : N/A
    GPU Virtualization Mode
        Virtualization Mode               : None
        Host VGPU Mode                    : N/A
        vGPU Heterogeneous Mode           : N/A
    GPU Reset Status
        Reset Required                    : Requested functionality has been deprecated
        Drain and Reset Recommended       : Requested functionality has been deprecated
    GPU Recovery Action                   : None
    GSP Firmware Version                  : 570.195.03
    IBMNPU
        Relaxed Ordering Mode             : N/A
    PCI
        Bus                               : 0x01
        Device                            : 0x00
        Domain                            : 0x0000
        Base Classcode                    : 0x3
        Sub Classcode                     : 0x0
        Device Id                         : 0x220610DE
        Bus Id                            : 00000000:01:00.0
        Sub System Id                     : 0x40401458
        GPU Link Info
            PCIe Generation
                Max                       : 4
                Current                   : 2
                Device Current            : 2
                Device Max                : 4
                Host Max                  : 5
            Link Width
                Max                       : 16x
                Current                   : 16x
        Bridge Chip
            Type                          : N/A
            Firmware                      : N/A
        Replays Since Reset               : 0
        Replay Number Rollovers           : 0
        Tx Throughput                     : 1130224 KB/s
        Rx Throughput                     : 23242 KB/s
        Atomic Caps Outbound              : N/A
        Atomic Caps Inbound               : N/A
    Fan Speed                             : 84 %
    Performance State                     : P5
    Clocks Event Reasons
        Idle                              : Active
        Applications Clocks Setting       : Not Active
        SW Power Cap                      : Not Active
        HW Slowdown                       : Not Active
            HW Thermal Slowdown           : Not Active
            HW Power Brake Slowdown       : Not Active
        Sync Boost                        : Not Active
        SW Thermal Slowdown               : Not Active
        Display Clock Setting             : Not Active
    Sparse Operation Mode                 : N/A
    FB Memory Usage
        Total                             : 10240 MiB
        Reserved                          : 389 MiB
        Used                              : 3385 MiB
        Free                              : 6468 MiB
    BAR1 Memory Usage
        Total                             : 256 MiB
        Used                              : 39 MiB
        Free                              : 217 MiB
    Conf Compute Protected Memory Usage
        Total                             : 0 MiB
        Used                              : 0 MiB
        Free                              : 0 MiB
    Compute Mode                          : Default
    Utilization
        GPU                               : 46 %
        Memory                            : 27 %
        Encoder                           : 0 %
        Decoder                           : 0 %
        JPEG                              : 0 %
        OFA                               : 0 %
    Encoder Stats
        Active Sessions                   : 0
        Average FPS                       : 0
        Average Latency                   : 0
    FBC Stats
        Active Sessions                   : 0
        Average FPS                       : 0
        Average Latency                   : 0
    DRAM Encryption Mode
        Current                           : N/A
        Pending                           : N/A
    ECC Mode
        Current                           : N/A
        Pending                           : N/A
    ECC Errors
        Volatile
            SRAM Correctable              : N/A
            SRAM Uncorrectable Parity     : N/A
            SRAM Uncorrectable SEC-DED    : N/A
            DRAM Correctable              : N/A
            DRAM Uncorrectable            : N/A
        Aggregate
            SRAM Correctable              : N/A
            SRAM Uncorrectable Parity     : N/A
            SRAM Uncorrectable SEC-DED    : N/A
            DRAM Correctable              : N/A
            DRAM Uncorrectable            : N/A
            SRAM Threshold Exceeded       : N/A
        Aggregate Uncorrectable SRAM Sources
            SRAM L2                       : N/A
            SRAM SM                       : N/A
            SRAM Microcontroller          : N/A
            SRAM PCIE                     : N/A
            SRAM Other                    : N/A
    Retired Pages
        Single Bit ECC                    : N/A
        Double Bit ECC                    : N/A
        Pending Page Blacklist            : N/A
    Remapped Rows                         : N/A
    Temperature
        GPU Current Temp                  : 60 C
        GPU T.Limit Temp                  : N/A
        GPU Shutdown Temp                 : 98 C
        GPU Slowdown Temp                 : 95 C
        GPU Max Operating Temp            : 93 C
        GPU Target Temperature            : 83 C
        Memory Current Temp               : N/A
        Memory Max Operating Temp         : N/A
    GPU Power Readings
        Average Power Draw                : 57.35 W
        Instantaneous Power Draw          : 51.13 W
        Current Power Limit               : 340.00 W
        Requested Power Limit             : 340.00 W
        Default Power Limit               : 340.00 W
        Min Power Limit                   : 100.00 W
        Max Power Limit                   : 340.00 W
    GPU Memory Power Readings 
        Average Power Draw                : N/A
        Instantaneous Power Draw          : N/A
    Module Power Readings
        Average Power Draw                : N/A
        Instantaneous Power Draw          : N/A
        Current Power Limit               : N/A
        Requested Power Limit             : N/A
        Default Power Limit               : N/A
        Min Power Limit                   : N/A
        Max Power Limit                   : N/A
    Power Smoothing                       : N/A
    Workload Power Profiles
        Requested Profiles                : N/A
        Enforced Profiles                 : N/A
    Clocks
        Graphics                          : 240 MHz
        SM                                : 240 MHz
        Memory                            : 810 MHz
        Video                             : 930 MHz
    Applications Clocks
        Graphics                          : N/A
        Memory                            : N/A
    Default Applications Clocks
        Graphics                          : N/A
        Memory                            : N/A
    Deferred Clocks
        Memory                            : N/A
    Max Clocks
        Graphics                          : 2100 MHz
        SM                                : 2100 MHz
        Memory                            : 9501 MHz
        Video                             : 1950 MHz
    Max Customer Boost Clocks
        Graphics                          : N/A
    Clock Policy
        Auto Boost                        : N/A
        Auto Boost Default                : N/A
    Voltage
        Graphics                          : N/A
    Fabric
        State                             : N/A
        Status                            : N/A
        CliqueId                          : N/A
        ClusterUUID                       : N/A
        Health
            Bandwidth                     : N/A
            Route Recovery in progress    : N/A
            Route Unhealthy               : N/A
            Access Timeout Recovery       : N/A
    Processes
        GPU instance ID                   : N/A
        Compute instance ID               : N/A
        Process ID                        : 2833
            Type                          : G
            Name                          : ............./Xorg
            Used GPU Memory               : 2051 MiB
        GPU instance ID                   : N/A
        Compute instance ID               : N/A
        Process ID                        : 3082
            Type                          : G
            Name                          : ......../gnome-shell
            Used GPU Memory               : 303 MiB
        GPU instance ID                   : N/A
        Compute instance ID               : N/A
        Process ID                        : 4084
            Type                          : G
            Name                          : ......................../omniverse-launcher
            Used GPU Memory               : 164 MiB
        GPU instance ID                   : N/A
        Compute instance ID               : N/A
        Process ID                        : 5937
            Type                          : G
            Name                          : ..................../brave
            Used GPU Memory               : 124 MiB
        GPU instance ID                   : N/A
        Compute instance ID               : N/A
        Process ID                        : 21744
            Type                          : G
            Name                          : .............../code
            Used GPU Memory               : 423 MiB
        GPU instance ID                   : N/A
        Compute instance ID               : N/A
        Process ID                        : 2038046
            Type                          : G
            Name                          : rustdesk
            Used GPU Memory               : 44 MiB
        GPU instance ID                   : N/A
        Compute instance ID               : N/A
        Process ID                        : 2039904
            Type                          : G
            Name                          : ................./rustdesk
            Used GPU Memory               : 31 MiB
    Capabilities
        EGM                               : disabled

'
2025-10-28T07:11:11Z [603,976ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'osDistro' = 'ubuntu'
2025-10-28T07:11:11Z [603,978ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'osName' = '22.04.5 LTS (Jammy Jellyfish)'
2025-10-28T07:11:11Z [603,980ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'osVersion' = '22.04.5'
2025-10-28T07:11:11Z [603,982ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'portableMode' = '0'
2025-10-28T07:11:11Z [603,983ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'primaryDisplayRes' = '1920x1080x32bit@60Hz'
2025-10-28T07:11:11Z [603,985ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'runEnvironment' = 'Individual'
2025-10-28T07:11:11Z [603,987ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'runningInContainer' = '0'
2025-10-28T07:11:11Z [603,989ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'shaderdb_debugSymbols' = '0'
2025-10-28T07:11:11Z [603,991ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'shaderdb_dumpIncludeOverrides' = '0'
2025-10-28T07:11:11Z [603,993ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'shaderdb_dumpIntermediates' = '0'
2025-10-28T07:11:11Z [603,996ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'shaderdb_obfuscateCode' = '1'
2025-10-28T07:11:11Z [603,999ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'shaderdb_optimizationLevel' = '1'
2025-10-28T07:11:11Z [604,002ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'systemInfo' = '
|---------------------------------------------------------------------------------------------|
| Driver Version: 570.195.03    | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
|     |                                  |        |     |            | Device-ID | UUID       |
|     |                                  |        |     |            | Bus-ID    |            |
|---------------------------------------------------------------------------------------------|
| 0   | NVIDIA GeForce RTX 3080          | Yes: 0 |     | 10486   MB | 10de      | 0          |
|     |                                  |        |     |            | 2206      | dbf1aa89.. |
|     |                                  |        |     |            | 1         |            |
|=============================================================================================|
| OS: 22.04.5 LTS (Jammy Jellyfish) ubuntu, Version: 22.04.5, Kernel: 6.8.0-85-generic
| XServer Vendor: The X.Org Foundation, XServer Version: 12101004 (1.21.1.4)
| Processor: 12th Gen Intel(R) Core(TM) i7-12700K
| Bare Metal Cores: 12 | Bare Metal Logical Cores: 24
| Available Cores:  20 
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 64039 | Free Memory: 48178
| Total Page/Swap (MB): 2047 | Free Page/Swap: 2047
|---------------------------------------------------------------------------------------------|
'
2025-10-28T07:11:11Z [604,004ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'telemetrySessionId' = '2188098965401816719'
2025-10-28T07:11:11Z [604,007ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'terminatedByAbort' = '0'
2025-10-28T07:11:11Z [604,010ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'totalRamBareMetalMB' = '64039'
2025-10-28T07:11:11Z [604,012ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'totalRamLimitedMB' = '64039'
2025-10-28T07:11:11Z [604,015ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'totalSwapBareMetalMB' = '2047'
2025-10-28T07:11:11Z [604,018ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'totalSwapLimitedMB' = '2047'
2025-10-28T07:11:11Z [604,020ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'userId' = 'default'
2025-10-28T07:11:11Z [604,023ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'workingDirectory' = '/home/<username>/IsaacSim/_build/linux-x86_64/release'
2025-10-28T07:11:11Z [604,026ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] Crash report files for upload:
2025-10-28T07:11:11Z [604,028ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'metadataFile' = '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.dmp.toml'
2025-10-28T07:11:11Z [604,031ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'upload_file_minidump' = '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.dmp'
2025-10-28T07:11:11Z [604,034ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]   'upload_file_traceback' = '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.py.txt'
2025-10-28T07:11:14Z [606,701ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] uploaded minidump: file: '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.dmp', code:200, response:
2025-10-28T07:11:14Z [606,709ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash]     4a26a6c0-b3cd-11f0-92ee-06dff9e1ac97
2025-10-28T07:11:14Z [606,713ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] Deleting file: '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.dmp.toml' (use setting "/crashreporter/preserveDump=true" to save)
2025-10-28T07:11:14Z [606,718ms] [Info] [carb.crashreporter-breakpad.plugin] attempting to delete the file '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.dmp.toml' with 10 retries remaining.
2025-10-28T07:11:14Z [606,726ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] Deleting file: '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.dmp'
2025-10-28T07:11:14Z [606,730ms] [Info] [carb.crashreporter-breakpad.plugin] attempting to delete the file '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.dmp' with 10 retries remaining.
2025-10-28T07:11:14Z [606,738ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] Deleting file: '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.dmp.toml'
2025-10-28T07:11:14Z [606,743ms] [Info] [carb.crashreporter-breakpad.plugin] attempting to delete the file '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.dmp.toml' with 10 retries remaining.
2025-10-28T07:11:14Z [606,747ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] Deleting file: '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.py.txt'
2025-10-28T07:11:14Z [606,752ms] [Info] [carb.crashreporter-breakpad.plugin] attempting to delete the file '/home/hri-david/.local/share/ov/data/Kit/Isaac-Sim Full/5.0/31a5493b-dd1f-4322-936ef490-c620a8c6.py.txt' with 10 retries remaining.
2025-10-28T07:11:14Z [606,828ms] [Error] [carb.crashreporter-breakpad.plugin] [crash]     dump file size is 0 bytes, file is readable.
Segmentation fault (core dumped)

Additional information

No response

[RandomOakForest]

Thank you for posting this. This issue occurs primarily due to CUDA memory access errors and changes in compatibility between USD Composer, Omniverse Warp, and Isaac Sim 5.0.0. The symptoms you described—successful execution in an older USD Composer but failure in Isaac Sim 5.0.0—indicate API changes or stricter device memory handling. The error cudaErrorIllegalAddress often results from device-side memory access violations, which can happen if Warp is passed arrays that are not fully compatible with its CUDA backend or when stage or mesh attributes are mutated incorrectly within Omniverse’s context.

Key Problem Areas

• OmniHydra Warnings: The warning about topology/point updates without corresponding normal updates suggests the rendering pipeline expects normals to be updated when points are deformed. Lack of this update can break appearance or even the underlying mesh data used for CUDA operations.
• Warp Memory: Passing arrays directly from USD to Warp, especially when using .Get() for mesh points, may introduce host-device transfer issues. The points may reside on the CPU, while the kernel expects them on the GPU, causing illegal memory access.
• Version Differences: Recent Isaac Sim versions (including 5.0.0) have stricter handling of USD and GPU resources. Operations that worked in older USD Composer or Isaac Sim versions are more likely to break if attribute or memory contracts are violated.

Solution Steps

1. Ensure Points Array is on the Correct Device

Before passing mesh points to Warp, explicitly transfer/convert them to a cuda device array. For example:

pointsarray = points.Get()
warparray = wp.array(pointsarray, dtype=wp.vec3, device="cuda", copy=True)
# Ensure 'copy=True' to force allocation and data transfer.

This helps avoid illegal address errors caused by trying to modify memory on the wrong device.

2. Update Normals After Deformation

After deforming mesh points, update the normals attribute to reflect new topology—this is especially important for proper rendering:

if prim.HasAttribute("normals"):
    # You need to recompute normals here as a Vt.Vec3fArray (refer to NVIDIA's documentation for examples)
    # For basic meshes: you can use Omniverse's geometry helpers or a custom function to calculate normals from updated mesh points
    new_normals = compute_normals(warparray.numpy()) # Replace with actual computation
    normals_attr = prim.GetAttribute("normals")
    normals_attr.Set(Vt.Vec3fArray(new_normals))

Not updating normals can trigger fallback warnings and disrupt rendering synchronization, potentially destabilizing memory layout for CUDA interop.

3. Check USD and Warp Compatibility

As of Isaac Sim 5.0.0, confirm you are using Warp and USDRT versions that match Omniverse’s recommendations. Sometimes overlapping usdrt/warp imports can introduce ABI issues. Stick to documented usage in the latest Isaac Sim documentation:

• Only use one set of USD/RT imports matching the authored scene.
• Call wp.init() before launching any kernel.

4. Use Omniverse Example Documentation

Test out NVIDIA’s published Warp mesh deformation samples in Isaac Sim 5.0.0. The latest documentation often provides guidance for safe memory and USD attribute manipulation that avoids direct host->device pitfalls:

• Review Isaac Sim Warp Integration Guide (5.0.0)
• Refer to Warp mesh editing documentation for up-to-date kernel signatures and mesh access patterns

Recommendations for Soft Robot Deformation

For robust and stable real-time deformation in Isaac Sim or Isaac Lab:

• Use the official Omniverse Physics-based deformation samples if available in the latest Isaac Sim release.
• Leverage the omni.physx interface for soft body and actuator logic—these APIs handle device memory and mesh attributes robustly and scale well with Isaac Lab.
• Avoid direct attribute mutation unless the Omniverse documentation confirms compatibility for that method.