Understanding project_pix_vjp and quat_to_rotmat_vjp.

[lukkio88]

Hello,

I was checking your code for the derivatives/jacobians in the helpers.cuh.
I think there's a miss understanding on my side on some of the calculations being done.

First of all here: project_pix

In my understanding this function performs the projection of a 3D point in image space. its counterpart for the jacobian is project_pix_vjp. This latter function given the jacobian of the loss with respect to the projected point it will calculate (using the derivative chain rule) the jacobian of the loss w.r.t. the the point before being projected.

Second quat_to_rotmat

Performs a conversion from a non unit quaternion to a rotation matrix and quat_to_rotmat_vjp calculates the jacobian of the Loss w.r.t to this quaternion (which again is non unitary since you normalize it inside the function) given the Loss w.r.t. the rotation matrix.

From some calculations I did for the project_pix_vjp this line

float4 v_proj = {
        v_ndc.x * rw, v_ndc.y * rw, 0., -(v_ndc.x + v_ndc.y) * rw * rw
    };

Does not seem correct to me the correct one I think should be

float4 v_proj = {
        v_ndc.x * rw, v_ndc.y * rw, 0., -(v_ndc.x*p_hom.x + v_ndc.y*p_hom.y) * rw * rw
    };

which seems also intuitively more consistent to me given the fourth component of v_proj has rw*rw factor rather than rw like the first two.

For the quat_to_rotmat_vjp from some calculations I did I think:

1. Since the quaternion in input is not normalized you should include the jacobian of the normalization
2. Also in the assignment of v_quat.x,...,v_quat.w some signs don't match. At the beginning I thought maybe because of column major storage I don't get the same signs as you do but actually some of them match the calculations I did.

My final questions are:

1. Are these calculations taken from the original gsplat implementation?
2. You have a derivation of these calculations?
   I can share mine if you want, I just don't want to clutter the github issue.