Thanks Ryan and Jenny for pushing me into the right direction.
I tried something out, but it did not work correctly. Here is my thought process as I am not sure where the error could be, hopefully you can spot something wrong:
First I created a child class of SceneCapture2D and added a GetPerspectiveViewMatrix() like this:
FMatrix NewPerspectiveMatrix = FPerspectiveMatrix(45, 512, 512, 10, 5000);
FMatrix NewViewMatrix = FLookAtMatrix(GetActorLocation(), GetActorForwardVector() + GetActorLocation(), FVector(0.0f, 0.0f, 1.0f));
FMatrix PerspectiveViewMatrix = NewPerspectiveMatrix * NewViewMatrix;
For now I hardcoded the values for the construction of the perspective matrix. And here are my first questions:
1. Is the near clip plane of a scene capture 2d the same value as the one we define under the project settings/ rendering tab?
2. Is the target point for the FLookAtMatrix constructor sufficient as ‘ActorForward + ActorLocation’?
Then I created a blueprint of my custom scene capture actor and filled 4 vector material parameters to use them in the material editor. In the material editor I multiply my PerspectiveViewMatrix with the position of the pixel:
3. Is my assumpion correct that a vector3 is not in row mode?
4. Is my matrix multiplication wrong?
The x and y components of my multiplication should give me values on the clip space plane. Then I need to adjust these values to stay in the uv of my custom depth texture:
My next step is now to convert the value of the depth texture back to my real depth value. I know that UE uses reserved z rendering, so I assumed to subtract the texture from 1 and then multiply it with my far plane value.
5. I assumed the depth is linear distributed. Is that correct? How do I reconstruct the depth value?
Then I would compare the distance of the scenecapture and the pixel with the depth value.
That didn’t work.
Hopefully someone has a clue. Thanks again for helping.