Unreal rendering frustum capabilities ?

Development Rendering
, ,
1

Currently, the two main questions are about :

  • Asymetric frustrum.

As far as we can see, a camera is defined using a view direction and viewing angles. Is there any way to have some asymetric projection ?

  • Far clipping plane

How is the far clipping distance used. We noticed impact on the Z buffer precision but content further away from the far distance is not actually clipped.

Is there any way to get it clipped (besides of course the atmosphere, …)

Thanks,

[Attachment Removed]

2

Steps to Reproduce
Hi,

We are working on integrating the CIGI standard ( https://cigi.sourceforge.io/ ) capabilities within Unreal and we have a couple challenges with regards to the visualization frustum.

Can you provide any insights on the approach used within Unreal ?

Thanks,

Basile

[Attachment Removed]

3

Hi Basile,

I am not entirely familiar with the CIGI standard, but here are my suggestions based on a bit of research I did in the last few days:

1. Is there any way to have some asymmetric projection ?

Yes, that should be possible, but you will need to make some custom engine changes and thoroughly test them. I was told you could try to do either one of the following options:

a) OffCenterProjectionOffset on FMinimalViewInfo (CameraTypes.h:129, units = proportion of screen dims). After the symmetric matrix is built, it rewrites the skew terms directly:

M[2][0] = (Left+Right)/(Left-Right); M[2][1] = (Bottom+Top)/(Bottom-Top); at CameraStackTypes.cpp:354-372. This shifts the frustum center off-axis (lens-shift style). You could also look at our nDisplay ICVFX inner-frustum implementation, which we use (DisplayClusterProjectionCameraPolicy.cpp:140-146) for inspiration.

b) Implement a fully custom projection matrix. The entire camera path produces only FSceneViewProjectionData::ProjectionMatrix, which the renderer consumes verbatim. You can bypass FOV+offset and feed a true general off-axis matrix (independent left/right/bottom/top) via a FSceneViewExtension (SetupViewProjectionMatrix hook) or a custom PlayerCameraManager/ULocalPlayer.

For the CIGI standard, you are better off with option B, since the independent edge angles need a custom projection matrix. You should be aware, though, that several subsystems in the renderer read the offset/FOV fields back (TAA jitter, DOF, etc.), so a raw matrix override can introduce some unforeseen side effects.

2. Far clipping

You are correct that we don’t actually clip geometry at the distance. You could attempt to do the following to clip far geometry:

- Supply your own FReversedZPerspectiveMatrix with MinZ != MaxZ. Geometry that gets past MaxZ gets NDC z<0 and is GPU depth-clipped, but you give up the infinite-far precision distribution and must manage the matrix yourself.

- Create a global clip plane by setting FSceneView::GlobalClippingPlane (SceneView.h:1803) via a SceneViewExtension and turning on r.AllowGlobalClipPlane (RendererSettings.h:993, member bGlobalClipPlane). That clips a half-space (a plane), which is exactly a flat far plane perpendicular to the view.

I hope that is enough info to get you started, but if you need any further clarification, please let me know.

[Attachment Removed]

4

Hi Tim,

1. Projection:

At the moment, we have a custom camera component which makes the projection. It implements (computes the offset) something similar to the ndisplay reference you mention.

I was indeed hoping for the user provided projection and was fearing the side effect you mentioned.

That being said, the OffCenterProjectionOffset might be enough. Looking into the formula used in openGL ( glFrustum : PyOpenGL 3.1.4 GL Man Pages ) I do not see other matrix elements that should be overriden.

On top of the renderer effects, it there any other aspects that should be double checked should we go with a user provided matrix ? In my past experiences (other engine), any view vector based based operation (billboard orientation for instance) was sensitive to this. Depth reconstruction, … any frustum culling ?

2. Clipping:

I guess we will have to try. Providing a clip plane seems acceptable and is not a priority since at the moment users have more content in view compared to a hard clipped behavior, …

However, the side question to this will be the relationship between near, far clip distances and Z fighting. We did see the “usual” Z buffer precision varying with the near / far ratio but I am not sure how this gets managed within the engine. Are you linearizing the Z buffer (logarithmic approach) or ??

Thanks,

Basile

[Attachment Removed]

5

Hi Basile,

On top of the renderer effects, it there any other aspects that should be double checked should we go with a user provided matrix ?

TAA and TSR jitter are the ones to watch, and they are also the main reason I would steer you toward OffCenterProjectionOffset. HackAddTemporalAAProjectionJitter adds into the same M[2][0] and M[2][1] terms, so a camera-level offset and the jitter coexist correctly. What breaks is overwriting the ProjectionMatrix wholesale from a hook that runs after jitter is applied, which silently removes anti-aliasing without any error or warning.

Are you linearizing the Z buffer (logarithmic approach) or ??

We are not linearizing the depth buffer, and it is not a log depth buffer. The log-like distribution you are seeing is reversed Z combined with a float depth target: float mantissa density is highest near 0, and under reversed Z.

That’s about as much as I have for you in terms of what I know about any potential side effects. Since you are making these custom engine changes to a very basic part of the engine, you will likely need to thoroughly test them. I hope that helps, but let me know if you need any further clarification.

Cheers,

Tim

[Attachment Removed]

6

Hi Tim,

I am back from vacation and will likely look back into this once I settle down and upgrade to 5.8.1.

Did you have a chance to review my comment regarding my attempt to add a clip plane ? I guess this should / could be straight forward to resolve.

Thanks,

Cheers,

Basile

[Attachment Removed]

7

Hi Basile,

Yes, that should be fine. Let me know if you run into any trouble while implementing, and I can try to give you some pointers.

Best,

Tim

[Attachment Removed]

8

Hi Tim,

The important line in the comment is :

I tried various values to the normal / point for the plane and in *NO* situation I was able to see any clipping happening. :frowning:

See the #pragma. I ensure the lines are compiled / executed.

Would you have any sample to share for this or engine source code line to double check by adding a breakpoint ?

I tested this only in PIE (Preview In Editor). Is there a chance this is restricted to packaged application ?

Thanks,

[Attachment Removed]

9

Hi Basile,

Sorry, we do not have any samples for this. Since you are overriding ISceneViewExtension::SetupView(), you can find the call sites in the engine and set breakpoints there, or check SceneView.cpp around line 2808 for the code that sets the View Uniform Buffer for the GPU passes. That way, you can step through with a debugger to figure out if your clipping plane is getting used properly by the engine. Otherwise, you could also try to get a RenderDoc or Pix capture to quickly verify that the clipping plane made it into the SceneView UniformBuffer.

[Attachment Removed]

10

Hi Tim,

[Image Removed]

It seems something else in our extension was interacting with the clipping plane definition. Not sure what at this point but I will figure this out with the developper who initiated it.

That being said, using an empty extension, I can get something to happen but this is not exactly what I would be hoping for…

  1. Can you confirm that the clipping plane is expected in world space ?

It will then belong to us to move the plane according to the eye point movement. That is quite accessible I suppose.

However, from the above image, the clip applies to the atmosphere / skybox. See the black layer with comments. That would be a show stopper for the feature.

2. Is there anything we can do to prevent the atmosphere / skybox from being clipped ?

I guess the fog is a separate rendering pass as it gets applied on clipped areas…

Thanks,

Basile

[Attachment Removed]

11

Hi Tim,

While researching other thing upgrading our application to Unreal 5.8.1, I fell on the below code implementation: (C:\Program Files\Epic Games\UE_5.8\Engine\Plugins\Runtime\nDisplay\Source\DisplayCluster\Public\Math)

/**

* Builds a perspective projection matrix from asymmetric near-plane extents (off-axis frustum).

* Inputs are signed distances from the eye to each frustum edge, measured at the near plane.

* The result uses Unreal’s Z-inverted left-handed convention.

*

* @param InLeft Left edge extent at the near plane (typically negative).

* @param InRight Right edge extent at the near plane (typically positive).

* @param InTop Top edge extent at the near plane (typically positive).

* @param InBottom Bottom edge extent at the near plane (typically negative).

* @param InZNear Near clip plane distance (must be > 0).

* @param InZFar Far clip plane distance. Pass InZNear to use an unlimited far plane.

*/

template <typename T>

static UE::Math::TMatrix<T> MakeAsymmetricProjectionMatrix(const T InLeft, const T InRight, const T InTop, const T InBottom, const T InZNear, const T InZFar)

{

const T mx = static_cast<T>(2) * InZNear / (InRight - InLeft);

const T my = static_cast<T>(2) * InZNear / (InTop - InBottom);

const T ma = -(InRight + InLeft) / (InRight - InLeft);

const T mb = -(InTop + InBottom) / (InTop - InBottom);

// Support unlimited far plane (InZFar == InZNear)

const T mc = (InZFar == InZNear) ? (static_cast<T>(1) - Z_PRECISION) : (InZFar / (InZFar - InZNear));

const T md = (InZFar == InZNear) ? (-InZNear * (static_cast<T>(1) - Z_PRECISION)) : (-(InZFar * InZNear) / (InZFar - InZNear));

const T me = static_cast<T>(1);

// Normal LHS

const UE::Math::TMatrix<T> ProjectionMatrix = UE::Math::TMatrix<T>(

FPlane(mx, 0, 0, 0),

FPlane(0, my, 0, 0),

FPlane(ma, mb, mc, me),

FPlane(0, 0, md, 0));

// Invert Z-axis (UE uses Z-inverted LHS)

static const UE::Math::TMatrix<T> FlipZ = UE::Math::TMatrix<T>(

FPlane(1, 0, 0, 0),

FPlane(0, 1, 0, 0),

FPlane(0, 0, -1, 0),

FPlane(0, 0, 1, 1));

return ProjectionMatrix * FlipZ;

}

Based on the comments, it would be capable of handling both my need at the same time using an easy toggle for the hard / infinite far clip plane.

Are you familiar with this code addition ? At first sight, this seems consistent with the CameraStackTypes code lines you first pointed me to.

Thanks,

Basile

[Attachment Removed]

12

Hi Basile,

I have not seen this code yet, but I am going to escalate the ticket to the code owner for an answer. They should be able to give you more advice on how to use the code.

Cheers,

Tim

[Attachment Removed]

13

Hi Tim,

For completeness and to keep you posted.

On the asymetric front :

  • I compared the matrix generation result from this code and the one from DesiredView.CalculateProjectionMatrix();. As far as I can tell, they are strictly identical. This approach is however allowing me to disable the infinite far capability and expose this as a user parameter.
  • I validated the result within our scene capture based view which has the nice “bUseCustomProjectionMatrix” member so my matrix can be used as it.

What remains to be done is :

  • Apply the custom matrix to a “split screen” based view. I need to investigate the extension / player approach. Last I checked, the call backs were not allowing me retrieve which one of our view was being rendered which was not making this straight forward. Need to look back at it. By the way, it lead to a separate question / bug on viewport application with and without antialiasing ( Split screen being clipped )
  • Understand how to prevent the skybox from being clipped. Assuming I understand it right, the skybox is a traditional geometry which is clipped as any other (either from the extra clip plane approach or from the hard far distance implementation in custom projection matrix).

On this point, you are probably in a good position to help me…

Is there any way to flag the skybox so it does *not* get clipped ?

Any accessible option ?

In another engine, we were overriding the depth output to 99.99% from the far distance so it was always visible. I could not find a way to do this in Unreal. I notice some “is sky” or similar check box on the material but that does not seem related.

Worse case, is there any resources that could demonstrate how to extend the pixel shader to do so.

Cheers,

Basile

[Attachment Removed]

14

Hi Basile, with regards to you SkyBox question: I built a small test project to verify this end to end, and it works. Here is the method, no Engine changes needed.

Create a Material Parameter Collection

Add an MPC with two scalar parameters:

FarDistance

(cm) and

ProtectSky

(0 or 1). Update it each frame from game code so the sky material always agrees with the plane:

UMaterialParameterCollectionInstance* Inst = World->GetParameterCollectionInstance(Collection);

Inst->SetScalarParameterValue(TEXT(“FarDistance”), FarDistance);

Inst->SetScalarParameterValue(TEXT(“ProtectSky”), bProtectSky ? 1.0f : 0.0f);

Add the pull-in to your sky material’s World Position Offset

The idea: slide every dome vertex along its camera-to-vertex direction so it sits just inside the clip distance. Direction is preserved, so the rendered sky is unchanged. In material nodes:

Dir = Normalize(AbsoluteWorldPosition - CameraPositionWS)

Pulled = CameraPositionWS + Dir * (FarDistance * 0.95)

WPO = (Pulled - AbsoluteWorldPosition) * ProtectSky

FarDistance and ProtectSky come from CollectionParameter nodes. The 0.95 margin keeps the dome safely inside the plane. With ProtectSky at 0 the offset is zero and you get stock behavior back.

Notes and caveats

  • This works because the WPO is applied before the clip distance is computed in the vertex shader, so the dome genuinely avoids the clip. A pixel-level depth override cannot work here: clipping happens at rasterization, before any pixel shader runs. This is the UE equivalent of the “write depth at 99.99% of far” trick from your previous engine, done at the vertex stage instead.
  • The same pull-in also protects the dome if you go with the hard far plane in a custom projection matrix (finite MaxZ) instead of the clip plane, since either way the dome ends up inside the far distance.
  • It is only valid for sky shading that depends purely on view direction. If your dome material uses absolute world position for something distance based, that part will change.
  • Height fog and the SkyAtmosphere/VolumetricCloud full-screen passes are not geometry and are unaffected by the clip plane either way.

[Attachment Removed]

15

Hi Tim,

Before you have a chance to look into the later question, I made a quick test with the clipping.

I have a dummy extension which does nothing but assigning the mentioned clip plane :

[Image Removed]

I tried various values to the normal / point for the plane and in *NO* situation I was able to see any clipping happening. :frowning:

I applied the cvar you mentioned and restarted a couple times the editor. Ensured it is set :

[Image Removed]Any clue on what I could be missing ?

Thanks,

[Attachment Removed]

16

[Content removed]

Link to the other ticket I opened yesterday. Seems the cpoy / paste had only the support ticket title and not the link, …

[Attachment Removed]