Adds raycast implementation and tests #146

nvtw · 2025-04-11T14:47:30Z

# Conflicts: # mujoco_warp/_src/io.py # mujoco_warp/_src/types.py

nvtw · 2025-04-11T15:05:56Z

I assume the tests currently fail because wp.tile_argmin was only merged yesterday into warp and might not yet be in the warp build we are using. Will investigate more on Monday.

thowell · 2025-04-14T10:26:32Z

mujoco_warp/_src/io.py

  m.mesh_vertadr = wp.array(mjm.mesh_vertadr, dtype=wp.int32, ndim=1)
  m.mesh_vertnum = wp.array(mjm.mesh_vertnum, dtype=wp.int32, ndim=1)
  m.mesh_vert = wp.array(mjm.mesh_vert, dtype=wp.vec3, ndim=1)
+  m.mesh_faceadr = wp.array(mjm.mesh_faceadr, dtype=wp.int32, ndim=1)
+  m.mesh_face = wp.array(mjm.mesh_face, dtype=wp.int32, ndim=2)


since mesh_face is (nmeshface, 3) this array could be wp.array(mjm.mesh_face, dtype=wp.vec3i, ndim=1)

thowell · 2025-04-14T10:30:17Z

mujoco_warp/_src/ray_test.py

+class RayTest(absltest.TestCase):
+  def test_ray_nothing(self):
+    """Tests that ray returns -1 when nothing is hit."""
+    m, d, _, _ = test_util.fixture("ray.xml")


following the convention of other tests:

mjm, mjd, m, d = ...

then it is not necessary to call mujoco.mj_forward, mjwarp.put_model, and mjwarp.put_data below since these functions are called by test_util.fixture.

thowell · 2025-04-14T10:32:13Z

mujoco_warp/_src/ray_test.py

+
+  def test_ray_plane(self):
+    """Tests ray<>plane matches MuJoCo."""
+    m, d, _, _ = test_util.fixture("ray.xml")


mjm, mjd, m, d = ...

thowell · 2025-04-14T10:32:24Z

mujoco_warp/_src/ray_test.py

+
+  def test_ray_sphere(self):
+    """Tests ray<>sphere matches MuJoCo."""
+    m, d, _, _ = test_util.fixture("ray.xml")


mjm, mjd, m, d = ...

thowell · 2025-04-14T10:33:19Z

mujoco_warp/_src/ray_test.py

+
+  def test_ray_capsule(self):
+    """Tests ray<>capsule matches MuJoCo."""
+    m, d, _, _ = test_util.fixture("ray.xml")


mjm, mjd, m, d = ...

thowell · 2025-04-14T10:34:33Z

mujoco_warp/_src/ray_test.py

+
+  def test_ray_box(self):
+    """Tests ray<>box matches MuJoCo."""
+    m, d, _, _ = test_util.fixture("ray.xml")


mjm, mjd, m, d = ...

thowell · 2025-04-14T10:35:02Z

mujoco_warp/_src/ray_test.py

+    _assert_eq(dist_np, mj_dist, "dist")
+
+  def test_ray_mesh(self):
+    """Tests ray<>mesh matches MuJoCo."""


mjm, mjd, m, d = ...

thowell · 2025-04-14T10:35:50Z

mujoco_warp/_src/ray_test.py

+
+  def test_ray_geomgroup(self):
+    """Tests ray geomgroup filter."""
+    m, d, _, _ = test_util.fixture("ray.xml")


mjm, mjd, m, d = ...

thowell · 2025-04-14T10:36:33Z

mujoco_warp/_src/ray_test.py

+
+  def test_ray_flg_static(self):
+    """Tests ray flg_static filter."""
+    m, d, _, _ = test_util.fixture("ray.xml")


mjm, mjd, m, d = ...

thowell · 2025-04-14T10:36:51Z

mujoco_warp/_src/ray_test.py

+
+  def test_ray_bodyexclude(self):
+    """Tests ray bodyexclude filter."""
+    m, d, _, _ = test_util.fixture("ray.xml")


mjm, mjd, m, d = ...

thowell · 2025-04-14T10:41:00Z

mujoco_warp/_src/ray_test.py

+    dist_np = dist.numpy()[0][0]
+    _assert_eq(geomid_np, -1, "geom_id")
+    _assert_eq(dist_np, -1, "dist")
+


please add a test for transparent geoms. mjx reference: https://github.com/google-deepmind/mujoco/blob/8e7457d7c6c1cbda42ec39d2297d4f90b3396040/mjx/mujoco/mjx/_src/ray_test.py#L238

thowell · 2025-04-14T10:41:53Z

mujoco_warp/_src/types.py

@@ -576,6 +583,7 @@ class Model:
    geom_condim: contact dimensionality (1, 3, 4, 6)         (ngeom,)
    geom_bodyid: id of geom's body                           (ngeom,)
    geom_dataid: id of geom's mesh/hfield; -1: none          (ngeom,)
+    geom_group: geom group inclusion/exclusion mask          (ngeom, 1)


thowell · 2025-04-14T10:42:30Z

mujoco_warp/_src/types.py

@@ -788,6 +801,8 @@ class Model:
  mesh_vertadr: wp.array(dtype=wp.int32, ndim=1)
  mesh_vertnum: wp.array(dtype=wp.int32, ndim=1)
  mesh_vert: wp.array(dtype=wp.vec3, ndim=1)
+  mesh_faceadr: wp.array(dtype=wp.int32, ndim=1)
+  mesh_face: wp.array(dtype=wp.int32, ndim=2)


wp.array(dtype=wp.vec3i, ndim=1)

mmacklin · 2025-04-22T08:37:48Z

mujoco_warp/_src/ray.py

+    triangle.v2 = m.mesh_vert[vert_start + v_idx.z]
+
+    # Calculate intersection
+    dist = _ray_triangle(triangle, pnt, vec, basis)


Btw Warp has BVH accelerated versions of these routines: https://nvidia.github.io/warp/modules/functions.html#warp.mesh_query_ray, also distance and sign queries: https://nvidia.github.io/warp/modules/functions.html#warp.mesh_query_point.

nvtw · 2025-04-22T08:52:10Z

I don't understand why the ray unit tests fail on github. They succeed on my Linux and on my Windows machine.

erikfrey · 2025-04-23T20:41:27Z

@nvtw Is it a CPU vs GPU issue? Do you have a way to force CPU on one of your dev machines and rerun the tests?

thowell · 2025-04-29T11:55:45Z

@nvtw it should be possible to reproduce the test failure locally with #200

thowell · 2025-05-01T09:38:19Z

@erikfrey is it ok to add these .stl files? it looks like they are from https://github.com/google-deepmind/mujoco/tree/main/mjx/mujoco/mjx/test_data/meshes.

the alternative would be to define meshes directly in xml?

# Conflicts: # mujoco_warp/_src/io.py

thowell · 2025-05-05T18:52:37Z

request that would be useful for the touch sensor implementation #213:

update _ray_geom to follow the mujoco implementation api, something like:
ray_geom(pos: wp.vec3, mat: wp.mat33, size: wp.vec3, pnt: wp.vec3, vec: wp.vec3, geomtype: int) -> float

here is the mujoco reference for mju_rayGeom: https://github.com/google-deepmind/mujoco/blob/253abad210823254f0f238ef3ad4cc8d975372d3/src/engine/engine_ray.c#L818

here is the mujoco reference for mjSENS_TOUCH: https://github.com/google-deepmind/mujoco/blob/253abad210823254f0f238ef3ad4cc8d975372d3/src/engine/engine_sensor.c#L777 where mju_rayGeom is utilized

thanks!

erikfrey · 2025-05-09T19:41:55Z

@thowell yes it's OK to bring those meshes in.

# Conflicts: # mujoco_warp/_src/io.py # mujoco_warp/_src/types.py

…oo messy

nvtw · 2025-05-12T11:05:59Z

Should be all done now

thowell · 2025-05-12T11:14:19Z

mujoco_warp/_src/io.py

@@ -127,17 +127,29 @@ def put_model(mjm: mujoco.MjModel) -> types.Model:
    tile_beg = tile_corners[i]
    tile_end = mjm.nv if i == len(tile_corners) - 1 else tile_corners[i + 1]
    tiles.setdefault(tile_end - tile_beg, []).append(tile_beg)
+
+  # Handle empty tiles case


is this block of code still utilized?

thowell · 2025-05-12T11:18:51Z

mujoco_warp/test_data/meshes/pyramid.stl

is this file needed?

thowell · 2025-05-12T11:24:35Z

@nvtw please take a look at the comment about the block of code in io.py and the comment about the mesh file. thanks!

nvtw added 15 commits April 10, 2025 11:34

Initial draft of ray cast queries

18dfe90

Add missing test files

f65d95a

Fix a couple of errors

92ed8d5

Fix more warp compatibility issues

848b6fb

Adapting the tests to warp

2a90a61

Tetrahedron test working

fba0207

Forgot to save a file

bc44ac5

Add early out

62f9fd5

Fix dodecahedron test

b2038cb

Started to introduce geomgroup

62077a4

First draft of filter logic

daa4241

test_ray_flg_static and test_ray_bodyexclude start to work

0a301cd

All tests pass

a8d58c9

Merge branch 'main' into dev/tw/ray_casting

b1397fc

# Conflicts: # mujoco_warp/_src/io.py # mujoco_warp/_src/types.py

Fix linter issues

815fd29

thowell linked an issue Apr 11, 2025 that may be closed by this pull request

Ray casting. #94

Closed

thowell reviewed Apr 14, 2025

View reviewed changes

nvtw added 5 commits April 14, 2025 14:41

Make _ray_box more similar to mujoco

279a5e4

Follow the original even more closely

6cf2bf6

Add test test_ray_invisible

bf609ec

Rename ray_geom to ray

844cf8a

Merge branch 'main' into dev/tw/ray_casting

e666462

mmacklin reviewed Apr 22, 2025

View reviewed changes

nvtw added 3 commits April 29, 2025 16:01

Merge branch 'main' into dev/tw/ray_casting

8b3e6fd

Merge branch 'main' into dev/tw/ray_casting

2587108

Fixed cpu implementation

0cd1b17

Merge branch 'main' into dev/tw/ray_casting

6dbc655

# Conflicts: # mujoco_warp/_src/io.py

nvtw added 7 commits May 12, 2025 10:21

Merge branch 'main' into dev/tw/ray_casting

af389a6

# Conflicts: # mujoco_warp/_src/io.py # mujoco_warp/_src/types.py

Reset io.py and types.py to the state of master since the merge was t…

3531da5

…oo messy

Bring back vec6 type

7e1b50c

Avoid using full model and data classes in ray casting

1eb27b5

Fix tests

0388b72

Fix style issues

697b593

Add overload for _ray_geom as requested in MR

1b41b2e

thowell reviewed May 12, 2025

View reviewed changes

mujoco_warp/test_data/meshes/pyramid.stl Outdated

Copy link

Collaborator

thowell May 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this file needed?

Removed unused stl file

b8bd380

thowell approved these changes May 12, 2025

View reviewed changes

thowell merged commit cd6bfbb into google-deepmind:main May 12, 2025
5 checks passed

Adds raycast implementation and tests #146

Adds raycast implementation and tests #146

Uh oh!

Conversation

nvtw commented Apr 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nvtw commented Apr 11, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nvtw commented Apr 22, 2025

Uh oh!

erikfrey commented Apr 23, 2025

Uh oh!

thowell commented Apr 29, 2025

Uh oh!

thowell commented May 1, 2025

Uh oh!

thowell commented May 5, 2025

Uh oh!

erikfrey commented May 9, 2025

Uh oh!

nvtw commented May 12, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thowell commented May 12, 2025

Uh oh!

Uh oh!

Uh oh!

nvtw commented Apr 11, 2025 •

edited

Loading