I figured something out. I’m not accepting my answer because I haven’t figured out how/when any of the built in matrices work.

As everyone probably knows, UE4 is really hard to customize. UE4 Graphics programming using the engine code, is not only harder, it’s also undocumented.

I tried all matrices both on the HLSL usf side and on the c++ side to no avail. So I made my own matrix from scratch every frame and uploaded it to the shader as a 4x4 `FMatrix`

to a shader via the UE4 `DECLARE_UNIFORM_BUFFER_STRUCT_MEMBER()`

macro system.

I render the camera to a quad. This means that in my Vertex shader I can treat it as if it is already in View space (instead of object space aka local space) to make it be exactly the size of the camera. So to get a ray in worldspace from that, I have to multiply by the Inverse Transpose View matrix.

Here’s how to construct an Inverse Transpose View matrix in UE4 in c++:

First get the Viewport of the client (the game running on your machine):

```
APlayerCameraManager* Manager = World->GetFirstPlayerController()->PlayerCameraManager;
ULocalPlayer* LocalPlayer = World->GetFirstLocalPlayerFromController();
FSceneViewFamilyContext ViewFamily(FSceneViewFamily::ConstructionValues(
LocalPlayer->ViewportClient->Viewport,
World->Scene,
LocalPlayer->ViewportClient->EngineShowFlags)
.SetRealtimeUpdate(true));
FVector ViewLocation;
FRotator ViewRotation;
const FSceneView* SceneView = LocalPlayer->CalcSceneView(&ViewFamily, /*out*/ ViewLocation, /*out*/ ViewRotation, LocalPlayer->ViewportClient->Viewport);
```

We need the `ViewLocation`

(the current player’s rendering camera’s position in world space) and the `ViewRotation.Vector()`

which is the camera’s forward vector. You can get these directly from the camera manager without `CalcSceneView`

, but I added the SceneView code for reference because it provides a lot of matrix resources (none of which I could get working).

Now just make a simple reverse View Matrix:

```
FMatrix FPixelShaderUsageClass::GetViewMatrix(FVector eye, FVector target, FVector up)
{
FVector forwardX = eye - target;
forwardX.Normalize();
FVector rightY = FVector::CrossProduct(forwardX, up);
rightY.Normalize();
//upZ is already normalized because it's a cross product of two normalized vectors
FVector upZ = FVector::CrossProduct(forwardX, rightY);
//The inverse and transpose of the View matrix.
return *(new FMatrix(
*(new FPlane(forwardX.X, rightY.X, upZ.X, eye.X)),
*(new FPlane(forwardX.Y, rightY.Y, upZ.Y, eye.Y)),
*(new FPlane(forwardX.Z, rightY.Z, upZ.Z, eye.Z)),
*(new FPlane(0, 0, 0, 1))
));
}
```

From this you can get a float4x4 matrix in your hlsl shader which you can mul with your vertex pos to go from “viewspace” to world space.