FAQ: Rendering

FAQ includes content originally written for UDN.

General:

What are best practices for extending the rendering engine, preferably in a maintainable way?

  • Avoid changing engine code and do everything you can in a plugin or project module. If there isn’t an API for a rendering change you want to make, work with us to add one.**

  • What is the best way to switch between render techniques in application when using forward Rendering for VR but not all players are not exclusively using VR, such as PC?**

  • Each device renders its own contents, not the other device’s contents, so you can make the decision based on which device the client is running on. The server never renders anything.**

Any simple examples of CustomMeshFactories with simple data flow? in engine or otherwise?

  • LidarPointCloud is probably the main example for a plugin, and otherwise you can look at the vertex factories contained in the engine**

What is the typical movie rendering workflow?

  • Render Movie Queue (4.25) allows you to do batch rendering on a render farm or headless rendering on your local machine. Currently, we have the infrastructure for anyone (engineers) to integrate Render Movie Queue with their render farm managing system. For 4.26 we are planning to integrate Deadline out of the box.**

  • Render (Remote) option launches an external process that renders all sequences in your queue. You must save your changes in the project so the external process can read the files from disk.**

  • The Render Remote option also allows you to implement a remote render farm while still preserving in-editor renders for quick previews while you adjust settings. The default behavior of the render options are determined by the Project Settings and can be adjusted to run your own code, such as adapting Render (Remote) to use third party render farm management software.**

  • Additionally, you have cvar commands which can be a starting place for building your own automated render farms.**

  • With the Render Movie queue, you can process denoising techniques and remove Temporal Anti-Aliasing. You can subsampling, override games settings and Adjusting cvars like:**

  • r.AmbientOcclusion.Denoiser.TemporalAccumulation 0**

  • r.GlobalIllumination.Denoiser.TemporalAccumulation 0**

  • r.Reflections.Denoiser.TemporalAccumulation 0**

  • r.Shadow.Denoiser.TemporalAccumulation 0

  • For now the output formats are: bmp sequence, exr sequence, jpeg sequence, png sequence, audio wav.

Rendering Features

We’ve watched the video on Destructible HLODs from GDC2019. Fortnite has evolved a lot since then. Does that approach still hold up? Any new learnings since then?

  • No real changes to destruction HLODs since GDC.

  • Only merged HLODs are destructible in Fortnite (HLOD0), not proxy HLODs (HLOD1)

  • We add vertex color to HLODs, use colors to assign object ids to vertices, and in materials we use vis buffer to toggle primitive on/off**

  • Geometry swap: works the same way, hide, then update vis buffer, if everything is static, you could then just show the swap pieces, allow HLOD to show geometry (merge HLOD, not proxy)

  • More texture sampling for vertexes, not reducing vertices, but reducing triangles

  • General Chaos Destruction documentation

What is the recommended practice to integrate HLODs into an active development workflow where assets are changing often?

  • 4.25 onwards has optimizations for HLOD building, 10x faster depending on content

  • There are some known issues with some clustering options, single cluster per level is extremely fast. We can provide some versions that didn’t make it to 4.25

  • Recommended practice is to have a process that rebuilds invalidated HLODs, gather list of changed HLODs, then split workload across multiple machines

What happens when a level is checked out?

  • Data is stored in level itself, so it skips updates on that one specific

  • We are in the process of trying to move HLOD data out of the level to reduce this cost

Due to our dynamic lighting setup, we are currently relying on CSMs for shadowing in order to accommodate destructible architecture. Once CSM fades out, we lack a shadow mechanism for midground and background. We also lack any type of AO solution other than post process material, but this has many drawbacks. Are there any potential mechanisms that can help us in terms of shadows and AO? Any suggested solutions we can implement ourselves if no other options currently exist?

  • ES3.1 is our current minimum, we couldn’t support distance field shadows yet for mobile previously because we needed the compute shader to merge the distance fields. We want to try it, but we don’t have any performance data yet. Also a memory tradeoff.

  • We feel like reducing the update frequency and quality to get it running on mobile.

  • Right now we don’t have any meta to add it, but in Fortnite the tech artists did some tricks for fake shaders for shadows.

  • AO - techniques to implement

  • Get other platforms distance based techniques

  • Fake it

  • Shaders walk through the same data for both, which is why neither are implemented yet

  • Is it an option to enable SSAO? (has some performance hits)

  • We implemented it locally, but has not yet been submitted, still in progress

  • Performance was acceptable, but does have side-effects: rendered after base pass so the effect is not very accurate

  • We cannot do temporal filtering since we don’t keep previous frame data

  • Implemented in 4.26

We’d like the same colors that are in the texture viewer for a video source to show up in the 3D scene without any alteration. We are aware of lighting and tone mapping, so we tried this by using the Unlit material or the Composure plugin, but we didn’t get the desired result. Any tips for making this work?

  • As for color, we don’t have a mechanism to have a tonemapper applied to a subset of the 3D scene. The way we work around that elsewhere is with Composure, which allows you to have separate layers for the media content and the 3D CG content. The tonemapper can be enabled or disabled per layer, which would allow video to have no tonemapper, while CG content would have the usual UE4 tone curve. This is the path we leverage, but if you’re having trouble making it work for your needs, let us know.

We are using “real-world” values for sun/sky brightness, which leads to physically correct surface luminosity and expected camera exposure values. These real-world values have a massive numerical range and seem to break some lighting components; are there settings that might not be obvious that we should change based on these lights?

  • Pre-exposure should be working and fix these precision issues. There were a few bugs that were fixed in 4.24 (like scene color nodes weren’t using pre-exposure) and there are some smaller ones that will be in 4.25 (skin shading specular). Pre-exposure should work consistently, but if not please report the bugs.

Currently building the reflection environment is editor only. What are the issues with doing this at runtime?

  • The ability to do it at runtime was removed to support SM4, but can be readded if you don’t care about SM4.

What other static lighting system can we use for objects that are placed procedurally and then don’t move again?

  • Lightmass is the only static lighting system and requires an editor-time bake of a level. That said, if your proceduralism is at cook time you could probably hook a system together to generate/bake them without manual intervention.

Our dynamically lit environments look good when there is a direct light on a surface, but our interiors are very flat when relying on indirect lighting. Is there a dynamic equivalent to volumetric lightmaps we should be using? Should we place “fake” dynamic lights indoors? Are there good content examples we could look at for dynamic indirect lighting for interiors?

  • UE4 only has a few possibilities for dynamic indirect. Distance Field AO can help if dynamic skylight is a main contributor to interior lighting (not usually the case). Light Propagation Volumes can also be used, but suffer from light bleeding and again, are best used outdoors. 4.24 added Screen Space GI which could add a nice effect. Otherwise you’re probably looking at a tech art solution of some kind of faked light if you have well controlled environments.**

We are using LPV for our dynamic global illumination. It works ok in most situations, but is not a complete feature, and our understanding is that further work is not in the roadmap. Are there improvements we should try to make ourselves?

  • It is correct that we are not doing further work on LPVs. We never had an internal roadmap for developing them so we don’t have a list of improvements to try on-hand, but you could likely make improvements based on having a specific use case to optimize for.

Is the new screen space GI in 4.24 a replacement for LPV? Is there a better dynamic GI solution we should be using?

  • SSGI is more of a supplement to LPV than a replacement, but you should definitely try it out. There is no better existing ‘dynamic’ GI solution that you should be using.

Does the engine provide a solution for multi-pass dithering, to achieve soft edges?

  • Such a system is not explicitly provided. There are dithering functions in the material graph that work fairly well with using TAA to fill the holes and provide a kind of transparent look. It may or may not work for this use-case. Depending on performance requirements you could probably write a multi-pass system like this in Composure.

Does Unreal have per-pixel displacement?

  • We have per-vertex displacement ‘World Position Offset’, and per-pixel depth modification (not the same as physical displacement) ‘Pixel Depth Offset’.

Does Unreal have mesh subdivision in the modeling tools?

  • As of 4.25, this functionality does not exist in UE, and there are no current plans to implement this.

How do you enable static rendering path for skeletal meshes?

  • In the skeletal mesh component details under the “Optimization” category there is a check box for “Render Static”. This sets the flag for marking the mesh for the static rendering path.

Why are “Exposure/metering mode: manual” and “auto_exposure_histogram” light values so different?

Do the light lumen values mean anything compared to real world values in auto_exposure_histogram?

  • No. No matter what EV the current view is, Auto_Exposure always ends up a proper exposure. For more information on Auto Exposure go to the following blog post

In auto_exposure_histogram do aperture, ISO and speed have any effect on the lighting or is it just to adjust the depth of field?

  • Aperture/ISO/Speed do nothing to exposure as of UE 4.25. Only DOF gets affected by Aperture.

What’s the roadmap for ray tracing support?

  • As of 4.24, PC ray tracing is ready for production, stable as possible, efficient, and feature complete.

  • We are working on adding ray tracing support for next-gen hardware, but this likely won’t be production ready until UE5 releases.

What are the plans around ray traced global illumination?

In your experience with ray-tracing, and outside teams implementing ray-tracing into their UE4 games, have you seen a method/procedure/formula emerge for taking a game from standard deferred rasterized rendering to ray-tracing?

  • There is not a single method we have seen emerging. Depending on your level of expertise you can follow different paths.

Rendering Performance:

If our modular pieces have LODs, is using Hierarchical Instanced Static Meshes the preferred instancing method?

  • UE can dynamically batch render instances. This is not perfect batching and HISM/ISM can result in improved performance depending on conditions. You will need to test which is best for your game.

What are the advantages/disadvantages of using Hierarchical Instanced Static Meshes over non-hierarchical?

  • HISM creates clusters of instances. Culling, occlusion, and ‘lodding’ is done on a per cluster basis for HISM but is done on the Actor for ISM.

Is the foliage system based on HISM, and is using the foliage system the only way to take advantage of HISM?

  • The foliage system is using HISM. You can aggregate instances in HISM manually or through the ‘Merge Actors’ tool in the editor. HISMs are just components as well, so you could also build your own systems on top of HISM.

Is the new custom per-instance data for Instance Static Meshes in 4.25 also supported in HISM?

  • Yes, it is supported on HISM.

Do we cover all of our rendering components with our thread safety guarantees?

  • Some of our components, including rendering, still have their state created on the game thread, so thread safety isn’t guaranteed, and would need to be worked on for these components. There may be some bugs in the implementation, but currently (as of 4.25) only 1 rendering component is entirely created on the rendering thread.

Do we remove render components from the render queue when we can determine that a render proxy is not needed for that component?

  • Probably not, since we still have to process the task generated by queuing the component between threads, so that could introduce some thread safety issues

For mesh decals or deferred decals, is it fine to use baked or dynamic lighting? Anything to watch out for?

  • For baked lighting, you need to use DBuffer decals, as they get rendered before the base pass (in the prepass). For dynamic lighting, either can be used.

  • We’re investigating deprecating the deferred decals so we only have one pass

How does UE4 best support high quality LOD0 for platforms of different power levels?

  • We only support different MinLOD for different platforms

  • Scalability changes the transition distances, but not the LODs themselves

Do we stream in textures out of the box?

  • Texture streaming should be enabled by default for textures that are not in the UI category, anything with mips

  • If you spend a lot of time in the streamer, you may be able to save 0.5-1 milliseconds by loading all mips ahead of time

For next-gen platforms, how does tessellation behave?

  • Probably just as bad as it is on current PCs, shouldn’t be worse but probably not better. We actively try not to work with it due to the heavy maintenance required to keep it up to date over time, especially when compared with the long term potential of Nanite in UE5.

In general what are the best practices for setting up the material architecture for our game?

  • Best practices will be hard to come up with here as it’s so very content dependent.

  • In general, you have to strike the right balance of features and flexibility without going overboard on instances. You’ll probably have master materials for each primary item type. Ideally customization would be through parameters like scalars, vectors, etc.

  • Avoid static switches when you can. Best to be mindful of creating permutations, lots of switches + usage flags creates many permutations. Usage flags being the killer, ie: having the same material usable on static meshes and skeletal should be avoided. This overflows very quickly if not managed.

  • Our work on the Material Layering system might help. You can make instances of layers and not cause a recompile/permutation if you swap in the same layer type.

Why was the texel density view removed from UE4?

  • This was removed as part of the purge when moving from UE3 to UE4 because it was burdened by too many legacy assumptions. It’s not likely that this feature will be added in the future.

Shadows are our biggest performance issue, since we cannot use lightmaps. Is there a way to optimize shadows for “static” objects at runtime?

  • There are limited tech options for shadows, especially directional lights.

  • Setting object mobility to ‘static’ will allow shadow caching for spotlights for example.

  • See these docs for for some more info on the rules.

  • Directional lights can switch to distance field shadows at distance further reducing the cost.

  • See this, this and mainly this

  • Furthermore you could experiment with some code changes - add shadow LOD bias (use lower LOD for shadow rendering) or add code to skip rendering of small objects into the shadow map.

We have many interiors that would benefit from pre-computed object visibility. Is there a way to group objects for optimized culling at runtime?

  • There’s a PVS system, but it may not be applicable depending on how procedural your world layout is. Details here.

Is there a work around for the hard limit on reflection captures?

  • There’s no existing workaround for reflection capture limits, but you could change the code.

What are your recommendations for an inexpensive depth of field for background objects?

  • The current depth of field has a good quality for cost tradeoff. You could in theory bring back the now mobile-only Gaussian DOF but sometimes it’s not actually faster. There’s also a ‘depth blur’ that simulates atmospheric blur embedded in the DOF that’s fairly cheap that runs even when things aren’t ‘out of focus’. You could try ramping that up.

  • r.DepthOfField.DepthBlur.Amount

  • r.DepthOfField.DepthBlur.Scale

  • r.DepthOfField.DepthBlur.ResolutionScale

What are good setups for various levels of pre-baking?

  • I think there’s some good answers in [this video class of ours.](Lighting with Unreal Engine Masterclass | Unreal Dev Day Montreal 2017 | Unreal Engine - YouTube]

  • Baking everything will be fastest. There are some good trade offs like stationary lights where you can change parameters of the light but not the position. Ray tracing would even be an option for next gen consoles, and we have support for that. LPV specifically is not a great solution because we never got it to a production ready state. Global Illumination that is performant is still an open challenge, but we do have a team exploring new options, and with the new features of the next generation of consoles, maybe it’ll be a bit more realistic of an option.

LightBaking: How would we bake small parts of a large world and have it blend with other baked (or unbaked) areas?

  • We don’t currently support baking selected actors only, but even if implemented, there would likely still be problems with blending. The best option at the moment would be always baking volumetric lightmaps and baking some surface lightmaps on some objects, letting the unbaked use the volumetric lightmap.**

SDF-AO vs baked worlds?

  • At the moment there’s no way to mix baked skylight occlusion (bent normal) with the one coming from the DFAO. We use one or the other based on skylight mobility. There are no fundamental limitations though and you can mix those two by modifying the engine.**

Localized ambient (“tent in a desert”-scenario)

  • If the tent is not movable and you can bake it into volumetric lightmaps or lightmaps then it could use bent normal. If it is movable, then it can use SSAO or DFAO

Decals with normal-falloff?

  • Decals use material graphs just like surface materials. Decals are capable of reading normals from the GBuffer and outputting properties based on that. Decals can also modify normals in the GBuffer.

Cascade shadows take a massive hit in quality once kicking them down from Epic to Medium scalability, are there any settings to slightly improve Medium?

  • The biggest difference between the high and medium scalability setting for shadows is that the number of cascades goes from 3 to 1. To improve visual quality, you may change the corresponding ini settings, of course being aware that this will come at a cost. This is configured for the engine in /Engine/Config/BaseScalability.ini. To override this on a per-project basis you can modify/create [ProjectDirectory]\Config\DefaultScalability.ini

  • To similarly override for a platform add [ProjectDirectory]/Config/[Platform]/[Platform]Scalability.ini

  • See these docs for ini file override rules.

What are the recommended anti-aliasing methods?

  • TAA is the expected anti-aliasing method for all UE4 rendering.

Is there any way around detail loss from TAA?

  • Inherent fundamental techniques of TAA can’t be avoided, but can be mitigated with higher detail content. TAA will eat subpixel features and is sensitive to very high contrast.**

Temporal AA is OK, but causes all kinds of weird ghosting and generally makes the screen look muddy. From your experience, have you found any tweaks to Temporal AA or using one of the others like MSAA that work better?

  • r.TemporalAA.HistoryScreenPercentage 200

  • The ghosting is likely happening when the input noise to TAA is too high. The command “vis SceneColor UV0” will show that.**

  • r.TemporalAA.HistoryScreenPercentage is expensive, but a technology that relies on Nyquist theorem to keep the sharpness of the detail as the frame gets reprojected in TAA. We’ve used that in many of our demos since The Reflection demo. This allows for instance to keep all the details of the skin as the character is constantly moving.

RHI

What are best practices for extending the rendering engine, preferably in a maintainable way?

  • Avoid changing engine code and do everything you can in a plugin or project module. If there isn’t an API for a rendering change you want to make, work with us to add one.

What is the best way to switch between render techniques in application when using forward Rendering for VR but not all players are not exclusively using VR, such as PC?

  • Each device renders its own contents, not the other device’s contents, so you can make the decision based on which device the client is running on. The server never renders anything.

Any simple examples of CustomMeshFactories with simple data flow? in engine or otherwise?

  • LidarPointCloud is probably the main example for a plugin, and otherwise you can look at the vertex factories contained in the engine**

What is the typical movie rendering workflow?

  • Render Movie Queue (4.25) allows you to do batch rendering on a render farm or headless rendering on your local machine. Currently, we have the infrastructure for anyone (engineers) to integrate Render Movie Queue with their render farm managing system. For 4.26 we are planning to integrate Deadline out of the box.

  • Render (Remote) option launches an external process that renders all sequences in your queue. You must save your changes in the project so the external process can read the files from disk.

  • The Render Remote option also allows you to implement a remote render farm while still preserving in-editor renders for quick previews while you adjust settings. The default behavior of the render options are determined by the Project Settings and can be adjusted to run your own code, such as adapting Render (Remote) to use third party render farm management software.

  • Additionally, you have cvar commands which can be a starting place for building your own automated render farms.

  • With Render Movie queue, you can process denoising techniques and remove Temporal Anti-Aliasing. You can subsampling, override games settings and Adjusting cvars like:

  • r.AmbientOcclusion.Denoiser.TemporalAccumulation 0

  • r.GlobalIllumination.Denoiser.TemporalAccumulation 0

  • r.Reflections.Denoiser.TemporalAccumulation 0

  • r.Shadow.Denoiser.TemporalAccumulation 0

  • For now the output formats are: bmp sequence, exr sequence, jpeg sequence, png sequence, audio wav.

Mobile Rendering

Is there a reference for the things like z-depth that are inconsistent in the viewport ES2 Preview and the device renderer?

  • Q2, 2020: Z-depth is different, with differences between Android and IOS

  • The framebuffer is used to help with this

  • We also use GL extensions, and iOS can’t fetch depth

  • We render a 16 bit depth into the alpha value

  • In 4.26, we changed how we calculate depth to provide more precision. Might have strange effects at far off objects

What limitations should we be aware of for instancing on Android/iOS?

  • When instancing using components like HISM, there are no limitations on iOS.

  • On Android, there are a few things to be aware of with Mali GPU based devices: firstly devices based on the Midgard architecture (Mali-T8xx series and earlier), there are limitations when about the range of adjacent vertex indices when using 16-bit index buffers, as some data is repurposed for instancing purposes. Rendering such meshes with instancing will result in corrupted triangles. 4.25 added a workaround r.Android.MaliT8Bug=1 which will detect content with this issue and expand them to 32-bit index buffers at load time.

  • Secondly, Mali GPUs run the vertex shader once and store the results in a parameter buffer. Then each screen tile is processed in turn running the fragment shader. The parameter buffer has limited size and the full render pipeline needs to be flushed and restarted when it fills up. In normal rendering this can be dealt with, but in the case of instancing, it’s possible to overflow this buffer when you are rendering millions of triangles in a single draw call. When that happens, some of your instanced meshes will fail to render. There is no mechanism to detect this situation and not much that can be done to mitigate it except breaking up your draw calls. We don’t see it in practice with Fortnite foliage but we have seen it with synthetic test scenes.

  • For auto-instancing we have the GPU Scene feature which is supported on mobile in UE 4.24 and UE 4.25, but mobile GPUs are not well optimized for the random buffer access patterns we use to look up vertex data and so this is slower than no instancing for regular geometry, and use of HISM for special cases like foliage in typical game scenes.

  • We have also seen rendering corruption with this feature on some old Mali T8xx based mobile devices which we believe are due to driver bugs.

Best practices with Layered Materials? Mobile limitations?

  • No limitation we know of with layered materials on mobile. They are just too expensive to use on mobile, so we never use them internally.

  • The mobile hardware sampler limit is 16 samplers.

  • Several textures are used by the landscape system for blending layers together (1 texture for the first two layers plus the normal map, and 1 extra texture for each 4 layers you apply).

  • The mobile renderer also makes use of a couple of textures: the CSM shadowmap if using dynamic shadows, or otherwise 2 textures for the precomputed light/shadowmaps, plus the reflection environment or skylight.

  • With all of that, 5 or 6 layers sounds about right. Limiting the number of layers per component is key here as that’s the important number. There’s a landscape visualization mode that shows that.

Is there a way to disable features which are known to be unrealistic to use on mobile?

  • UE4 has the mobile renderer which is tuned to specifically ship mobile titles for lower end markets. The mobile renderer in UE4 currently defaults to the minimum specs used in Fortnite, but those can be modified by dozens (if not hundreds) of console variables or using the device profiles.

See more about Rendering on the Knowledge Base.

3 Likes