Basic Help in Writing a Custom Plugin

[Disgruntoad]

Hi all, I'm attempting to write a PragueSunSky plugin that can represent wavelengths from 280 --> 2480nm (https://github.com/PetrVevoda/pragueskymodel). I've converted the base code to use DrJit function calls and data structures, primarily dynamic arrays. I've confirmed the function calls reproduce the original code's output.

I'm now attempting to implement the code in Mitsuba, but I'm a little lost on where to begin. I read through https://mitsuba.readthedocs.io/en/stable/src/developer_guide/writing_plugin.html, but it's not clear to me (a) what functions are necessary for an emitter plugin (I assume eval(), eval_direction() and pdf_direction()?) and (b) what the arguments to these functions need to be? I'm also not entirely sure what the shape of Spectrum is for spectral and scalar rendering. I assume it's a tensor of [height, width, channels]?

Is this documented somewhere or what would be the best way for me to understand Mitsuba's code flow? Where are calls to emitter plugins located in Mitsuba's rendering code?

Thank you for the help!

[njroussel]

Hi @Disgruntoad

Here's a few useful links:

• The full interface of an Emitter than you can override in your custom implementation: https://mitsuba.readthedocs.io/en/stable/src/api_reference.html#mitsuba.EmitterPtr
• I'd suggest you have a look at the source code of existing emitter plugins to have a better idea of what typically needs to be implemented: https://github.com/mitsuba-renderer/mitsuba3/tree/master/src/emitters
• You can also look at the full interface of emitters: https://github.com/mitsuba-renderer/mitsuba3/blob/master/include/mitsuba/render/emitter.h#L53 (the mapping of C++ to Python types should almost always be one-to-one)
• For the Spectrum type you can read more about how it gets instantiated here: https://mitsuba.readthedocs.io/en/stable/src/developer_guide/variants_cpp.html#variants-in-c
• The uppermost relevant piece of code that will call your emitter usually is the integrator. You'll want to look up the source code of the one you're planning to use to understand exactly which interfaces it needs.

[njroussel]

I'm going to assume that this was written out of simplicity, rather than "ideal" mitsuba code. It indeed seems a bit overly intricate, but it's just a test.

In spectral variants, a single ray will always be evaluated for 4 wavelengths at a time (this is somewhat arbitrary). Hence the Array4f types you've seen. For SurfaceInteraction3f, as you've seen it's also used in some evaluations (emitters and sensors notably). We therefore need to store the wavelenghts that the ray had on the si object too, so we can pass that information on. Typically, each ray will have 4 unique wavelenghts - having 4 of the same is just a waste.

[Disgruntoad]

Awesome, thanks for the clarification!

[matttsss]

If I can just add a little precision on the tests, the wavelengths are loaded in this fashion in order to evaluate multiple wavelengths in one Dr.Jit kernel. Here the wavelenghts are repeated for convenience of initialization, that approach is proper to the tests of the sun model since there is only one ray and multiple wavelengths to test.

This approach is not used though in the eval_full_spec since we have both multiple rays (for the whole sky) and multiple wavelengths. Hence if we truly wanted to evaluate everything in one kernel, we would have needed to create a mesh grid on the rays and wavelengths for the dimensions to match ^^

[Disgruntoad]

Hi @matttsss @njroussel ,

This may be due to my ignorance of C++ templates / drjit dispatching, but I'm confused on how data is passed around, specifically as used in sunsky.h eval() and the test_sunsky.py render_and_compare().

When I evaluate the python code, si.wi is a matrix of (3, [image_size]). However, eval() declares that local_wo = func(si.wi) is a Vector3f type. Is Vector3f able to store arrays implicitly? Or is dr jit dispatching eval() image_size times on each 3-element vector in parallel (this seems to be what the documentation implies)? Or is local_wo somehow a pointer to an array of Vector3f data? It doesn't seem to be a pointer based on the declaration.

If the eval() function is dispatched n times in parallel, where / how is this done in the code? I am confused that SurfaceInteraction is an argument and also somehow its wi attribute would be dispatched properly?

[njroussel]

Hi @Disgruntoad

You should to read this page and this one too. It goes over how Dr.Jit launches computation and some trivial examples about the arrays & types you find in Dr.Jit.

In short, almost every object is vectorized (it's an array). Operations on theses objects are not executed immediately, they are first recorded by Dr.Jit which then creates a kernel that actually executes the operations, but this kernel is only launched as late as possible. You do not need to care about how the kernel actually parallelizes your operations. Once the result is computed, it is stored in memory so that it can be easily referred to in the future.