Loading Disney PBR Material Model from nvdiffrec repository

[jensenzhoujh]

Hi @njroussel ,

In NVIDIA's nvdiffrec paper, they also used the Disney PBR model. I am wondering whether we can seamlessly migrate the PBR defined in their paper to Mitsuba 3. According to the released code for the shading function in this repo, I tried to load the object defined like this:

# diffuse reflectance
kd = np.array(Image.open(osp.join(dirname, 'texture_kd.png')), dtype=np.float32) / 255.
kd_bitmap = mi.Bitmap(kd, channel_names=['R', 'G', 'B'])
kd_bitmap.set_srgb_gamma(True)
kd_bitmap = mi.Bitmap(kd_bitmap).convert(
    pixel_format=mi.Bitmap.PixelFormat.RGB,
    component_format=mi.Struct.Type.Float32,
    srgb_gamma=False,
)
kd = np.array(kd_bitmap)
reflectance = { 'type': 'bitmap', 'bitmap': kd_bitmap, 'raw': True }

# specular reflectance
krom = np.array(Image.open(osp.join(dirname, 'texture_ks.png')), dtype=np.float32) / 255.
roughness = krom[..., 1]; metalness = krom[..., 2:3]
specular_reflectance = {'type': 'bitmap', 'bitmap': mi.Bitmap( (1 - metalness) / (metalness + 1e-6) * 0.04 + kd ), 'raw': True }
roughness = {'type': 'bitmap', 'bitmap': mi.Bitmap( roughness ), 'raw': True }
weight = {'type': 'bitmap', 'bitmap': mi.Bitmap( metalness[..., 0] ), 'raw': True }

# normal map
normal_bitmap = np.array(Image.open(osp.join(dirname, 'texture_n.png')), dtype=np.float32) / 255.
normal = {'type': 'bitmap', 'bitmap': mi.Bitmap( normal_bitmap ), 'raw': True }

object = mi.load_dict({
    'type': 'obj',
    'filename': mesh_path,
    'bsdf': {
        'type': 'normalmap',
        'normalmap': normal,
        'bsdf': {
            'type': 'blendbsdf',
            'weight': weight,
            'diffuse': {
                'type': 'diffuse',
                'reflectance': reflectance,
            },
            'roughconductor': {
                'type': 'roughconductor',
                'specular_reflectance': specular_reflectance,
                'distribution': 'ggx',
                'alpha': roughness,
            },
        },
    },
})

But it didn't end up getting me the right rendering. Is it possible for loading materials from them directly or not given the implementation difference?

[merlinND]

Hello @shallowtoil,

If the parameters correspond to the Disney BSDF, you should be able to use the principled BSDF plugin: https://github.com/mitsuba-renderer/mitsuba3/blob/6a1644335d9e9564afa6858456e19970706cefb8/src/bsdfs/principled.cpp
Then it's just a matter of loading and arranging the parameters in a dict like you've done.

[tangjm24]

Sorry to Bother you, have you figured out how to migrate the PBR defined in nvdiffrec to mitsuba3. I am also struggle with it. Thanks...