Almost got it.
• Unrotate the vector from target to camera (for ease of calculation, not needing camera up/right vectors).
• Take the X component of the vector, which is now the distance from the camera plane, and from there, use trigonometry and the FOV of the camera to calculate the extents of the view port in relation to said vector.
Effectively projecting the extents of the viewport to camera space.
• Divide the vector’s Y and Z coordinates by the horizontal and vertical extents respectively.
That should return the coordinates in widget/view space, mapped from -1 to 1.
• [OPTIONAL] Divide by 2, to get coordinates from -0.5 to 0.5.
The horizontal coordinate works perfectly, but for some reason I have an offset for the vertical coordinate, and I can’t seem to figure it out.
I tried scaling the screen to the widget aspect ratio is 1 to 1, and I get some weird behavior.
The Z (vertical) coordinate is half of what it should be, and has what appears to be some linear scaling offset.
The offset would’ve made sense normally, a mistake in calculating the aspect ratio, but when the aspect is 1:1 it just leaves me scratching my head.
The scale issue of the vertical coordinate is a mystery to me.
The camera ratio is not the texture ratio, I checked that.
Here’s the graph so far.
Note the vertical coordinate is multiplied by -1, while the horizontal is multiplied by 0.5.
No idea what’s going on with the vertical coordinate.
