Hi Unreal experts
I am trying to achieve a pixel (or near pixel) accurate “object picking” feature in our game. The center of the screen needs to sample an “Object ID buffer” to see what object is being rendered at that pixel. I can’t use collision/tracing for this because it only hits the bounding volumes - not enough accuracy - and I don’t want to turn on per-poly collisions since we don’t need that for all the objects in the game (it would be way to inefficient as well). This is just for object picking/selection.
So instead I am investigating reading back/sampling from a render-buffer to discover what object is being rendered at the sampled pixel.
I know there is a “visibility buffer” (from the View Modes) that looks like it could work, but I don’t know if it stores the object IDs, or is it just storing color? I couldn’t find any docs that explain exactly what is stored in this buffer.
If the visibility buffer doesn’t store object IDs, then I would be more than happy to write the object IDs into a new, custom channel in the gbuffer. We are going to use custom shaders for rendering the world, so adding some lines to write the object ID is no trouble.
However, I don’t know how to extend the gbuffer to store additional data. Are there official docs on this? Or would it involve altering the Unreal Engine source code? (Would like to avoid that if possible).
I am also wondering how one can sample the visibilty or gbuffer from C++. I presumably need to lock the buffer, get a pointer to the pixel data, and read it. Or maybe I’d have to copy the buffer, and then read the pixel data from the copy - so as not to stall the GPU rendering pipeline. Or something like this - but I have no idea where to start reading or learning how to do this kind of thing in Unreal.
Any tips/help greatly appreciated!
Marcus