Hit testing gpu-offset mesh.

Lots of meshes in my map have have vertex position offset their shader.
I want to get the color of these meshes under the player’s cursor (on the cpu).

You would think i can just get the pixel color at the cursor position. However there is a catch; I need to get the color of all meshes under the cursor, Including ones that are occluded by other meshes.

Does anyone have any ideas on how this could be done? One idea I have, is to make a row of hundreds of cameras under the player cursor. Each would render a single pixel at a different point in 3d space. However this is not a very performance friendly or accurate solution.