@plangton
Your case is indeed much more complicated than I thought. I don’t know how to get the data from the depth texture. I thought you were trying to make stuff in 3D appear as if it was behind stuff in real life (I did it for a virtual wardrobe with Kinect 1 so the person could be in front of the 3D objects).
I have an idea. Don’t know if it would work, though. You could “hack” into the depth texture from making a C++ function that accepts a UTexture2D as parameter and making it accessible in Blueprints, so you would be able to pass the texture into your C++ function. After that, you could use the pixel at the coordinate in X and Y pixels to calculate the X and Y cm based on the camera field of view and that pixel distance from the camera. I don’t thing that’s an easy formula, but I’m sure there’s trigonometry to do it.