Goal
Guys, I want to develop a function to display the view of Camera/SceneCaptureComponent, just like the function that CityEngine has, please refer to this image:
I have some thoughts but can not finish it yet, really need some suggestions.
Edit:
The Project is quite lightweight, you can have a try
My Thoughts
My idea is Setting a SceneCaptureComponent2D, and set the Capture Source to SceneDepth, then set the Texture Target to a RenderTarget2D. Then calculate the ViewProjection matrix of the SceneCaptureComponent2D and pass it to a postprocess volume material. In the postprocess material’s custom node, it iterates over every pixel of the current view, convert the world position of the current pixel into the clipspace of the SceneCaptureComponent2D, then we can judge if the pixel is in the viewfrustum of the SceneCaptureComponent2D, and if the depth is bigger than the same position in the Captured Depth Texture, paint it red, otherwise paint it green.
Detailed Steps
- Create a SceneCapture2D in the scene, and set the Capture source to SceneDepth, and Texture Target to a RenderTarget2D, which is set to RTFR16F format;
- Create a PostProcess Material(Later added it to the prost process volume through c++);
- Create a MaterialInstanceDynamic from the PostProcess Material, and add it to the PostProcess Volume’s Material array
void AVisibilityView::BeginPlay()
{
Super::BeginPlay();
SceneCaptureComp = SceneCaptureActor.Get()->GetComponentByClass<USceneCaptureComponent2D>();
MatInstance = UMaterialInstanceDynamic::Create(PostProcessMaterial.Get(), this);
if (PostProcessVol.IsValid())
{
PostProcessVol->Settings.AddBlendable(MatInstance, 1.0f);
}
}
- In Tick, get the ViewProjectionMatrix of the SceneCapture, and pass it to the Postprocess Material through 4 Vectors
void AVisibilityView::Tick(float DeltaTime)
{
Super::Tick(DeltaTime);
if (SceneCaptureComp->IsValidLowLevel() && MatInstance->IsValidLowLevel())
{
FMatrix TempMatrix = UMyBlueprintFunctionLibrary::FindViewProjectionMatrix(SceneCaptureComp);
MatInstance->SetVectorParameterValue(FName("MatrixRow0"), FVector4(TempMatrix.M[0][0], TempMatrix.M[0][1], TempMatrix.M[0][2], TempMatrix.M[0][3]));
MatInstance->SetVectorParameterValue(FName("MatrixRow1"), FVector4(TempMatrix.M[1][0], TempMatrix.M[1][1], TempMatrix.M[1][2], TempMatrix.M[1][3]));
MatInstance->SetVectorParameterValue(FName("MatrixRow2"), FVector4(TempMatrix.M[2][0], TempMatrix.M[2][1], TempMatrix.M[2][2], TempMatrix.M[2][3]));
MatInstance->SetVectorParameterValue(FName("MatrixRow3"), FVector4(TempMatrix.M[3][0], TempMatrix.M[3][1], TempMatrix.M[3][2], TempMatrix.M[3][3]));
}
}
The code of function “FindViewProjectionMatrix”
FMatrix UMyBlueprintFunctionLibrary::FindViewProjectionMatrix(const USceneCaptureComponent2D* sceneCaptureComponent2D)
{
// View Matrix
const FTransform& transform = sceneCaptureComponent2D->GetComponentToWorld();
FMatrix viewMatrix = transform.ToInverseMatrixWithScale();
viewMatrix = viewMatrix * FMatrix(FPlane(0, 0, 1, 0), FPlane(1, 0, 0, 0), FPlane(0, 1, 0, 0), FPlane(0, 0, 0, 1));
// Projection Matrix
const float FOV = sceneCaptureComponent2D->FOVAngle * (float)PI / 360.0f;
FIntPoint captureSize(sceneCaptureComponent2D->TextureTarget->GetSurfaceWidth(), sceneCaptureComponent2D->TextureTarget->GetSurfaceHeight());
float XAxisMultiplier;
float YAxisMultiplier;
if (captureSize.X > captureSize.Y)
{
XAxisMultiplier = 1.0f;
YAxisMultiplier = captureSize.X / (float)captureSize.Y;
}
else
{
XAxisMultiplier = captureSize.Y / (float)captureSize.X;
YAxisMultiplier = 1.0f;
}
FMatrix projectionMatrix = FReversedZPerspectiveMatrix(FOV, FOV, XAxisMultiplier, YAxisMultiplier, GNearClippingPlane, GNearClippingPlane);
return viewMatrix * projectionMatrix;
}
- About the PostProcess material
The core is the Custom node here: The input “Tex” is fed with a Texture Object, which is fed with the RenderTarget2D; The input “WorldPosition” is fed with a WorldPosition node
Custom Node Input and Output Settings:
Custom Node Code:
float4 col0 = Col0;
float4 col1 = Col1;
float4 col2 = Col2;
float4 col3 = Col3;
float4x4 vpmatrix = float4x4(col0.r, col1.r, col2.r, col3.r,
col0.g, col1.g, col2.g, col3.g,
col0.b, col1.b, col2.b, col3.b,
col0.a, col1.a, col2.a, col3.a);
float3 worldPos = WorldPosition;
float4 clipspace = mul(worldPos, vpmatrix);
return (clipspace.x > -1
&& clipspace.x < 1
&& clipspace.y > -1
&& clipspace.y < 1
&& clipspace.z > 0
&& clipspace.z < 1) ? 1.0 : 0.2 ;
// Cannot produce the expected result, so stopped here
Question
You can see the Custom node code is not finished, because I can’t visualize the clipspace of the SceneCaptureComponent correctly, is there some problem when passing the ViewProjection Matrix?
