Z distance resets at some point

Hi,
I'm playing around with the z distance. If you get too close to the camera or too far away, the guessed z distance kind of resets itself and jumps to values like -150. Am I doing something wrong?

Here's my current example: https://vin-ni.github.io/z-to-em/

Thanks 👍
