Calculation of discontinuous derivatives of visibility

[ejtwe]

Summary

There is a difference between the render_primarily_visible_silhouette function in the common.py file and the derivative calculation in the paper.

Description

In the render_primarily_visible_silhouette function of common.py:

          def render_primarily_visible_silhouette(self,
                                                      scene: mi.Scene,
                                                      sensor: mi.Sensor,
                                                      sampler: mi.Sampler,
                                                      spp: int) -> mi.TensorXf:
                  """
                  Renders the primarily visible discontinuities.
          
                  This method returns the AD-attached image. The result must still be
                  traversed using one of the Dr.Jit functions to propagate gradients.
                  """
                  film = sensor.film()
                  aovs = self.aov_names()
          
                  # Explicit sampling to handle the primarily visible discontinuous derivative
                  with dr.suspend_grad():
                      # Get the viewpoint
                      sensor_center = sensor.world_transform() @ mi.Point3f(0)
          
                      # Sample silhouette point
                      ss = self.proj_detail.sample_primarily_visible_silhouette(
                          scene, sensor_center, sampler.next_2d(), True)
                      active = ss.is_valid() & (ss.pdf > 0)
          
                      # Jacobian (motion correction included)
                      J = self.proj_detail.perspective_sensor_jacobian(sensor, ss)
          
                      ΔL = self.proj_detail.eval_primary_silhouette_radiance_difference(
                          scene, sampler, ss, sensor_center, active=active)
                      active &= dr.any(dr.neq(ΔL, 0))
          
                  # ∂z/∂ⲡ * normal
                  si = dr.zeros(mi.SurfaceInteraction3f)
                  si.p = ss.p
                  si.prim_index = ss.prim_index
                  si.uv = ss.uv
                  p = ss.shape.differential_motion(dr.detach(si), active)
                  motion = dr.dot(p, ss.n)
          
                  # Compute the derivative
                  derivative = ΔL * motion * dr.rcp(ss.pdf) * J
                 ......

The relevant formulas in the paper are as follows：

---But I don't see the sinφ item.
---And p in "p = ss.shape.differential_motion(dr.detach(si), active); motion = dr.dot(p, ss.n)" is the barycenter coordinate?
Does it correspond to 𝜕𝜃 xb in the paper?
---Can the calculation of the Jacobian J provide a more detailed derivation or explanation? (function perspective_sensor_jacobian())

Thank you very much for your answer！

[ziyi-zhang]

Hi @ejtwe,

This part of the code is largely irrelevant to the projective paper. The paper focuses more on indirect effects in PBR, and this function renders the primarily visible derivatives which is an orthogonal discussion.

The computed p from differential_motion is the differential version of the intersection point, fully attached to the shape parameters (vertices for triangles, center and radius for spheres, etc). The motion of a point associated on a surface is parameter-dependent since the horizontal in-surface motion could be arbitrarily large or small depending on the surface parameterization. But fortunately the derivative only needs to normal direction motion, which is an intrinsic property and can be easily computed via dr.dot(p, ss.n).

The Jacobian in perspective_sensor_jacobian is manually derived and only works for perspective cameras. We plan to rewrite that Jacobian with autodiff shortly.

Best

[xxcisxxc]

I just kindly inquire that do you still plan to rewrite the Jacobian with autodiff currently?

[njroussel]

It's not a priority, but a nice-to-have for sure. It's not particularly complex, it's just annoyingly difficult without changing a lot mitsuba interfaces IIRC
.