Hit result under cursor for object - just not suitable for what Im trying to achive?

You use some grid system, calculate that brown rock position (in grid coordinates) then check what is there.