Thank you for writing this code. I was looking for something exactly like this.
When I mouse over a reference that's on the right side of the page, it still shows the entire page (unlike in the preview gif you have, where it only shows the right part of the page). If I zoom in and click the link, I'm taken to the correct place in both x and y.

For reference the PDF is https://arxiv.org/pdf/1711.07971.pdf and I'm on Firefox 107.0 (latest version as of this post)
Another difference to your gif is that the top left corner of the preview does not coincide with the mouse location. In both cases, the gif is the more desirable behavior.