Repeated Ray Tracing runs with identical seed produce non-negligible dB differences in path_gain (up to ~3.7 dB)

[whaot]

I am running Sionna's RadioMapSolver (version: 1.2.2) multiple times on the exact same scene, with the same configuration and same random seed (seed=42). I am computing a full 3D radio map (40 height layers).

While I expect bit-exact reproducibility in a deterministic framework, I am observing large differences in the dB domain (up to ~3.7 dB) across runs.
How should I configure the settings to ensure the results are absolutely reproducible?
ray_trace_compare.ipynb

Here is the .zip of the scene:
26.zip

[whaot]

After I set diffraction=False, this issue no longer occurs. Does the diffraction computation involve other random processes?

[merlinND]

Hello @whaot,

Thank you for sharing the notebook and scene in order to reproduce the issue.

While I expect bit-exact reproducibility in a deterministic framework

Sionna RT is not guaranteed to be deterministic (this has been discussed many times in this forum). Even with fixed randomness from the RNG, massively parallel execution implies that different threads finish their work at different times. The precise timing is random, which means that results are accumulated in a random order. And since IEEE floating point operations are not commutative, this also means that the result can vary slightly depending on the order of operations.

1. It seems to me like the errors occur mostly in regions where the signal strength is already low. Could you please check whether the differences between runs could simply be due to the numerical error mentioned above?
2. Could you please share a minimal reproducer that only involves simple built-in Sionna RT components? E.g. PyTorch should not be needed. In principle, the many radio map "layers" should not be needed either. Please make it as simple as possible so that isolating the issue is easier.

[whaot]

Thanks for your reply.

Reply to 1:
It seems that the errors all appear in wedge regions after diffraction, and the path gains there are roughly in the range of –80 to –110 dB.

Reply to 2:
This is a notebook that will compute the radio map for a single height layer only.
RayTrace_Consistency_Test_SingleLayer.ipynb

The computation uses only numpy and Sionna (with Mitsuba and Dr.JIT); PyTorch is used only to read a few scene parameters, such as the transmitter position.

What's more:

1. I suspect the root cause is the massively parallel execution. I have recently been reading sb_candidate_generator.py. My current hypothesis is that during the diffraction computation, a large number of rays are processed in parallel, and the buffer can only retain the results of a subset of them (Perhaps those that finish first, or selected based on hash collisions. I'm still learning the code now.)
   This means the order and timing of completed ray computations can vary from one run to another, leading to slightly different retained sets and thus different final path gains. Of course, this is just a guess at this stage. If this is indeed the case, increasing the buffer size for diffraction might reduce the variability.

2. It seems that the difference doesn't occur every time, even setting diffraction=Ture. I completed
   RayTrace_Consistency_Test.ipynb
   , it shows
   === Layer-by-Layer Error Report ===
   [*] Analyzing worst-case pair: Run 1 vs Run 3
   Layer | Max Error (dB) | Mean Error (dB) | Max Error Location (Y, X)
   Z = 0 | 0.000015 | 1.055106e-07 | (Y=0, X=218)
   Z = 1 | 0.000015 | 1.035349e-07 | (Y=10, X=298)
   Z = 2 | 3.066620 | 2.578404e-03 | (Y=111, X=204)

However, in RayTrace_Consistency_Test_SingleLayer.ipynb, it shows

Let me know if you have any further suggestions or need additional details.

[merlinND]

Hello @whaot,

Regarding your point (1), your observation about sb_candidate_generator.py is correct. That particular hash table is only used in PathSolver(), but there is indeed a similar mechanism that is used to de-duplicate diffraction wedges during radio map computation: a hash table with size RadioMapSolver._WEDGE_INTERSECTIONS_SIZE:

https://github.com/NVlabs/sionna-rt/blob/bc0549155c7b782c7614a0ec06a0ac4e32b979ae/src/sionna/rt/radio_map_solvers/radio_map_solver.py#L188-L189
https://github.com/NVlabs/sionna-rt/blob/bc0549155c7b782c7614a0ec06a0ac4e32b979ae/src/sionna/rt/radio_map_solvers/radio_map_solver.py#L526-L539

In your scene, the issue is not that this array runs out of space, but rather that the order in which entries are added to it affects subsequent random wedge sampling.
The wedge sampling would ideally only be controlled by the given seed, but in practice the random ordering in the wedges hashtable also affects it.

We will implement a fix that will be part of the next release.
Thanks again for reporting this issue!

[whaot]

Thank you for your reply.
I'm very glad to hear that you have identified the cause of the issue. Looking forward to seeing your solution in the near future.