procedural kitchens exported to urdf do not have textures?

[RussTedrake]

from scene_synthesizer.procedural_scenes import kitchen
scene = kitchen(seed=None)
scene.export('kitchen.urdf')

Gives me a wonderful array of kitchen environment geometries. Awesome!

It looks like all of the resulting assets have a simple default gray color, e.g.

    <visual name="refrigerator_shelf_4">
      <geometry>
        <box size="0.6223100385705346 0.5043856382790842 0.01"/>
      </geometry>
      <origin xyz="0.0 0.05 0.07500000000000001" rpy="0.0 -0.0 0.0"/>
      <material name="refrigerator_shelf_4_main_color">
        <color rgba="0.4 0.4 0.4 1.0"/>
      </material>

(the objs that get generated also don't have any materials specified)

I've seen textured scenes in your paper, e.g.:

Based on my first read of the code, I don't see those textures being created/attached during the scene creation anywhere from the kitchen gen entry point. Might you be able to advise me on how I can get them?

[RussTedrake]

I did find this, but understand that this workflow is specific to USD, correct?

I think the caption on the doc threw me? (top to bottom, then left to right made me think it was columnwise...) I guess center bottom must be Isaac Sim and the bottom right must be Isaac Lab?

[clemense]

Ah, yes, sorry for the confusion. The materials in the screenshots are USD-specific and shown in IsaacSim and IsaacLab.
There's - IMO - no straightforward mapping between these PBR-type materials to other formats (e.g. OBJs MTL), that's a bit beyond the current functionality of scene_synthesizer.

This doesn't mean you couldn't add some textures to the geometries - if you provide them. I'm not extremely knowledgeable with good texture datasets that could be used. One option would be Pixar 128 (review their license terms, but it's CC-BY). Using some of those textures one could for example do the following (note that i'm using the function get_geometries that i just added in version 1.14.0):

from scene_synthesizer import procedural_scenes as ps
from PIL import Image

# Assumption: the pixar 128 dataset is extracted to /data/
texture_wood = Image.open("/data/2846_pixar128_library/Pixar128 Library/wood/American_cherry_pxr128.tif")
texture_marble = Image.open("/data/2846_pixar128_library/Pixar128 Library/floor/Gray_swirl_marble_pxr128_bmp.tif")

# Create a random l-shaped kitchen
kit = ps.kitchen_l_shaped()

# To add textures to the geometries we first need to create UV maps (here using xatlas)
# Note that this also changes the primitives (boxes etc) to meshes
kit.unwrap_geometries()

# select all geometries that match the regular expression
# here: contains the word "cabinet" but doesn't contain countertop, sink or handle
for g in kit.get_geometries("^(?=.*cabinet)(?!.*countertop)(?!.*/sink)(?!.*handle).*"):
    # g is of type trimesh.Trimesh
    g.visual.material.image = texture_wood

# Do the same as above but select all geometries that represent countertops
for g in kit.get_geometries('.*countertop.*'):
    g.visual.material.image = texture_marble

kit.export('/tmp/kit.usd')
kit.export('/tmp/kit.urdf', use_absolute_mesh_paths=True, mesh_path_prefix='file://')

If I open these files in Isaac and RViz respectively they both contain the textures (doesn't look as good as the PBR materials though):

One could elaborate this further to capture all scene elements. Let me know if there's some functionality that I could add to make this process easier.