Discrepancy between paper and code in the normal consistency loss

[lopeLH]

In the original 2DGS paper, the normal consistency loss for a given pixel is defined as:

However, in the code, the loss is:

2d-gaussian-splatting/train.py

Line 83 in 335ad61

normal_error = (1 - (rend_normal * surf_normal).sum(dim=0))[None]

with surf_normal and rend_normal defined as:

2d-gaussian-splatting/gaussian_renderer/__init__.py

Lines 144 to 147 in 335ad61

	surf_normal = depth_to_normal(viewpoint_camera, surf_depth)
	surf_normal = surf_normal.permute(2,0,1)
	# remember to multiply with accum_alpha since render_normal is unnormalized.
	surf_normal = surf_normal * (render_alpha).detach()

2d-gaussian-splatting/gaussian_renderer/__init__.py

Lines 120 to 123 in 335ad61

	# get normal map
	# transform normal from view space to world space
	render_normal = allmap[2:5]
	render_normal = (render_normal.permute(1,2,0) @ (viewpoint_camera.world_view_transform[:3,:3].T)).permute(2,0,1)

If I understand correctly, this implies the following pixel consistency loss formula:

$\mathcal{L}_n = 1 - \big( \sum_i w_i n_i \big) ^{\top} \bigg( \lfloor \sum_i w_i \rfloor ~ N \bigg)$

where $N$ is surf_normal, $\sum_i w_i n_i$ is rend_normal, $\sum_i w_i$ is render_alpha, and $\lfloor \cdot \rfloor$ denotes the stop gradient operator.

However, this is not exactly the same as the formula in the paper.

Is this discrepancy intentional? What are the effects or differences in behaviour between these two formulas? It would be great to clarify this, since a lot of papers are adopting a similar loss term, inspired and citing 2DGS, but it's unclear if they are using the paper or the implementation here as the reference.

[hbb1]

The difference is very minor in my experiments. For most datasets, $w_i$ sums to 1, so there is no differences for the $N$. The implementation here is to make it more suitable for transparent dataset like NeRF, where the N is weighted by the same amount of opacity .