How to get the world coordinates of the mouse pointer position?

Development Programming & Scripting C++
1

For creating scene aware camera navigation I’m trying to get the world coordinates of the object that is displayed behind the mouse pointer. I already tried using DeprojectMousePositionToWorld to get the mouse direction and LineTraceSingleByChannel to trace a line in that direction. This worked, but it requires that all meshes have a collision mesh defined as well. I would like to find the position on the displayed mesh instead, and I expect that this could be efficiently achieved by sampling the depth buffer at that the mouse position. Is there an easy way to access the depth buffer? Or is it possible to trace a line over the displayed meshes instead of the collision meshes? Or is there maybe some other way for retrieving the world coordinates of whatever is displayed behind the mouse pointer?

2

Pretty sure if your trace parameters have “bTraceComplex” set to true then trace collides with the mesh and not just simplified collision box.


 FCollisionQueryParams rayTraceParameters(FName("Trace"), true);
 rayTraceParameters.bTraceComplex = true;
3

Also, looking at trace FHitResult it looks like there is an ImpactPoint and ImpactNormal. Never used them but might be what you are looking for?

4

Thank you for your reply. Even with bTraceComplex=true, the line trace did not work for objects without a collision mesh. In the end, I solved the problem in another way. I attached an extra SceneCapture2D component to the camera and let it render the scene depth to a render target. Afterwards, I sampled the image from that render target at the mouse position to get the scene depth. I only let it capture an image when I needed the depth so there is no overhead when it is not used.