Per-vertex attributes for a BSDF

[tangjm24]

Currently I have a triangle mesh scene, in which each vertices has other attributes like (normal, diffuse color, albeldo, roughness, metallic). How can I just use those attributes to define a bsdf dict in mitsuba3 without transforming those attributes to bitmaps first. Thanks for anyone's' help.

[njroussel]

Hi @tangjm24

You will want to have a look at the mesh_attribute plugin (documentation). That plugin allows you to define per-face or per-vertex attributes that you can then assign to any BSDF parameter that expects a texture.

[tangjm24]

Thank you for your quick response. I am a beginner in this area. I currently need to work with a relatively complex scene (in .ply format) where it is difficult to define textures and material maps through UV mapping. Therefore, I hope to define color and material information (diffuse color, roughness, metallic, albedo) in the .ply file, and then use these per-vertex attributes to define the BSDF material. I am only dealing with scenes that involve reflective surfaces. Could you please advise me on how to define BSDF in this case?

Best wishes.

[njroussel]

You'll want to combine the following BSDFs:

• principled for the albedo, roughness, "metallicness", ... (documentation)
• normalmap for the normals (documentation)

i.e something like:

mi.load_dict({
    'type': 'ply',
    'filename': 'path/to/ply/with/attributes.ply',
    'bsdf': {
        'type': 'normalmap',
        'normalmap': {
            'type': 'mesh_attribute',
            'name': 'vertex_normalmap'
        }
        'inner_bsdf': {
            'type': 'principled',
            'base_color': {
                'type': 'mesh_attribute',
                'name': 'vertex_albedo'
            },
            'metallic': {
                'type': 'mesh_attribute',
                'name': 'vertex_metallic'
            },
            'roughness': {
                'type': 'mesh_attribute',
                'name': 'vertex_roughness'
            },
            ...
        }
    }
}

[tangjm24]

Thank you for your patient reply. I have a general understanding of how to organize my code. Lastly, I would like to ask if using the "principled" BSDF is equivalent to or compatible with the Cook-Torrance microfacet models with ggx normal distribution. The BRDF properties I obtained are the result of inverse rendering using Cook-Torrance and GGX distribution on multiple views.

[njroussel]

The principled BSDF, as far as I remember, has an underlying GGX distribution. If I'm not mistaken there is some mapping between the roughness parameter and the alpha values you'd usually find in a GGX model.
You'll find more details in the author's work.

[tangjm24]

Sorry to bother you again, but I've encountered a very strange problem. I followed the method you provided for using mesh attributes and first attempted to use the colors from the original .ply file as the base color. However, I am unable to display the expected colors correctly during rendering.

Here is the .ply mesh in MeshLab:

Here is my code:

import mitsuba as mi
mi.set_variant("scalar_rgb")
from mitsuba import ScalarTransform4f as T
import numpy as np


import trimesh

import drjit as dr
from drjit.auto import Float, Array3f, UInt 


scene_dict = {  
    'type': 'scene',  
 
    'integrator': {  
        'type': 'path'  
    },  

    'emitter': {  
        'type': 'envmap',  
        'filename': 'envmap.exr', 
        'scale': 1.0  
    },  

    'ply_object': {  
        'type': 'ply',  

        'filename':"\path\to\ply\file", 
        'filename':'fixed_inplace.ply',
        'face_normals': False,  
        'bsdf': {
             'type': 'diffuse',
             'reflectance': {
                 'type': 'mesh_attribute',
                 'name': 'vertex_color'
             }
         }
    },
}  



scene = mi.load_dict(scene_dict)  

w2c_gs = np.array(
    [[-0.94422938, -0.14619733,  0.2950546 , -1.89741043],
       [ 0.23149349,  0.34252699,  0.91054161, -2.45799736],
       [-0.23418292,  0.92806336, -0.28958032,  3.86228406],
       [ 0.        ,  0.        ,  0.        ,  1.        ]]
)
gs2mi = np.array(
    [[-1,0,0,0],
    [0,-1,0,0],
    [0,0,1,0],
    [0,0,0,1]]
)
w2c_mi = w2c_gs@gs2mi
# c2w = np.linalg.inv(w2c_mi)
to_world = T(w2c_mi)
print(to_world)
# sensor_count = 6

sensor: mi.Sensor = mi.load_dict({
    "type": "perspective",
    "fov": 34.36,
    "to_world": to_world,
    'film': {  
        'type': 'hdrfilm',  
        'width': 3950,  
        'height': 2968,  
        'rfilter': {'type': 'gaussian'},  
        'pixel_format': 'rgb',
        
    }, 
})


image = mi.render(scene, spp=64, sensor=sensor)
mi.util.write_bitmap('output.png', image)  

[njroussel]

Is the second image rendered with Mitsuba ? It doesn't seem completely wrong. Is the original color data in sRGB space ? I'm fairly certain that the system expects them to be in linear space.