Optimize the size of an area light

[AnsonSavage]

Hi!

I'm learning Mitsuba and loving it! I am beyond impressed at the technical genius that has gone into this project.

I am running into a couple of newbie issues. I posted one of my questions on Stack Overflow: https://stackoverflow.com/questions/79431447/gradient-lost-when-constructing-a-matrix-in-mitsuba-3

I suppose a simpler pre-req to that question that I have is: instead of trying to optimize a latent variable that controls the area light's size, how come I can't optimize the area light's to_world matrix?

When I try to print the scene traversal, it seems to show that the to_world matrix is differentiable:

SceneParameters[
  -------------------------------------------------------------------------------------------------
  Name                                          Flags    Type           Parent
  -------------------------------------------------------------------------------------------------
  PerspectiveCamera.near_clip                            float          PerspectiveCamera
...
  Light.to_world                                ∂, D     Transform4f    Rectangle
...

If I do something like:

parameter_key = 'Light.to_world'
optimizer = mi.ad.Adam(lr=0.005)
optimizer[parameter_key] = scene_params[parameter_key]

I get this error:

Exception                                 Traceback (most recent call last)
Cell In[62], line 42
     40 parameter_key = 'Light.to_world'
     41 optimizer = mi.ad.Adam(lr=0.005)
---> 42 optimizer[parameter_key] = scene_params[parameter_key]
     43 scene_params.update()
...
Exception: Optimizer.__setitem__(): value should be differentiable!

I thought it was differentiable, but maybe I'm missing something?

If I instead do

parameter_key = 'Light.to_world'
optimizer = mi.ad.Adam(lr=0.005)
optimizer[parameter_key] = scene_params[parameter_key].matrix

To tell the optimizer to optimize the underlying matrix, I get no errors here, but I do get an error when I call

dr.backward(loss)

Stating that:

TypeError: backward_from(): the argument does not depend on the input variable(s) being differentiated. Raising an exception since this is usually indicative of a bug (for example, you may have forgotten to call dr.enable_grad(..)). If this is expected behavior, skip the call to backward_from(..) if dr.grad_enabled(..) returns False.

What am I missing?

Thanks!
Anson

UPDATE 2-15-2025
Okay, I've worked on this a little bit more, and it seems also that using params.keep() is having the same issue. Which makes sense. I just read the tutorial on the optimizer and that was super helpful. Now I'm going to just try and optimize the area light's vertex locations, if I can.

UPDATE 2-17-2025
Unfortunately it faces the same error. Zero gradient is propagated.

[Start1er]

Hi,
In my experience, the to_world matrix is non-differentiable directly. Because so many reasons, for example, the sillueate is incorrect when you directly estimate the matrix. Instead, you can estimate the rotation vector or translation vector. You may have some more in here

[AnsonSavage]

Hi @Start1er! Thank you for your answer, great point. I forgot to update this answer once I came to one conclusion. This is what I posted on the Stack Overflow version of this question:

"After trying three variations on the second approach that wasn't working, I was looking at this tutorial and discovered that by setting:

scene = mi.load_file("my_scene.xml", integrator=prb_projective')

instead of

scene = mi.load_file("my_scene.xml", integrator=prb')

my optimizer was again able to find a gradient with respect to the size of the light (in the case that worked for me, I was editing a latent variable that was used to control a scale matrix that positioned the vertices of the area light).

Although this works, there are a couple things that should be noted:

1. It's much slower to use a projective integrator
2. It's extremely unclear how Mitsuba was able to figure out how to adjust the size of the light in the first case when using the non-projective prb integrator, but why it wasn't able to in the second case until I switched to the prb_projective integrator.

Any thoughts on this would be greatly appreciated."