Coming over from the SVOGI thread and the Physical Lights thread I decided to make a new thread and see if there's any interest on this.
The point is an attempt to repurpose the Reflection Captures into Localized Image-Based Lighting.
The point is an attempt to repurpose the Reflection Captures into Localized Image-Based Lighting.
The expected result would be a setup of baked ambient lighting (without using Lightmass) that goes beyond the limits of a globally-affecting skylight, that can co-exist in a dynamic lighting setup.
The most important use case would be indoor-outdoor ambient light differentiation, such as going inside a house or cave and not having the skylight affecting it. This is a commonly requested feature that is so far not addressed.
Up to UE4 4.11 there was a hidden feature called DiffuseFromCaptures which apparently was a way to contribute to lighting via reflection captures but it felt very short at it, and was removed anyway.
Up to UE4 4.11 there was a hidden feature called DiffuseFromCaptures which apparently was a way to contribute to lighting via reflection captures but it felt very short at it, and was removed anyway.
So I started a proof of concept which turned into a feature
What this does
TL;DR - Reflection Captures (Box & Sphere) are repurposed and used as Image-Based Lighting probes, which provide local ambient light while "blocking out" the skylight - allowing proper interior lighting when using a dynamic skylight.

Some comparison shots:
https://imgsli.com/MTYzMg
https://imgsli.com/MTYzMw
https://imgsli.com/MTYzNA
https://imgsli.com/MTYzNQ
How it looks if you smoothly change the ReflectionCapture's contribution at runtime:

Test on a production-like environment: UT4's awesome Outpost23 map

Outpost23 comparison screenshots here: https://imgur.com/a/EjEJ2zd
And the classic Sponza Atrium scene

How to set up
The engine changes were meant to go into the engine as a Pull Request, -integrated and maintained by Epic- and available for everyone directly from the binary engine. Because who wouldn't like having an additional ambient lighting choice when working with dynamic lights?
Unfortunately the Pull Request was declined by Epic, pretty much coming down to this:
Unfortunately the Pull Request was declined by Epic, pretty much coming down to this:
This would add another option that has the potential to confuse artists about what is needed to achieve their visual goals.
While I understand this system is not a great fit for the engine's straightforward "Static / Stationary / Dynamic" mobility concept and the grasp of light-object interaction it produces, there are a number of things that don't fit that view either (i.e. translucency in general, screen-space reflections).
Moving forward the system is available to integrate manually, for which you'll need a custom UE4 build.
The code is available on my GitHub Fork, where I add branches corresponding to the major engine versions (ignore the 'master' and 'release' branches).
The expected workflow is to compare my fork with the original UE4 codebase, and then manually integrate the highlighted differences into your own build. I only update my branches to major engine versions, so if you use my fork directly you'll miss out on all engine hotfixes.
Supported versions are 4.20, 4.21, 4.22, 4.23, 4.24, 4.25 and 4.26.
Future Versions
I don't actively use the system and maintaining custom engine versions requires a too big amount of time.
With the announcement of UE5 and its Lumen system for GI I have decided to stop maintaining this system. I will support it until the last version of UE4 (4.26 or 4.27), but will not carry it over to UE5. Therefore if you are thinking about newly integrating this system I'd recommend you to look for other approaches.
For dynamic lighting and handling of indoor/outdoor ambient lighting here's some alternatives that might be suitable depending on your projects' needs:
How to use
- set r.LocalIBLFromCaptures to 1 via console (set to 0 to disable again if needed). you probably want to add this into the ConsoleVariables ini
- You'll need to re-compile all Translucent materials manually, since the flag is not propagated for those until the shader is compiled.
- Make sure you have a Skylight that is set to Movable
- Place ReflectionCapture Actors (sphere or box) encompassing the areas where you want Local IBL
- Build the Reflection Captures to get instant results
- Tweak your lights' "Indirect Lighting Intensity" to control how much each light contributes to the Local IBL (you probably should set it to 0.0 if the light will change during runtime). Build the Reflection Captures when finished
- Optionally tweak your PostProcessVolume's 'Indirect Lighting Intensity' and 'Indirect Lighting Color' (applies as global changes) if needed. Build the Reflection Captures when finished
- Optionally modify each ReflectionCapture's ContributionFactor value in cases where the full IBL isn't wanted (both at editor time and at runtime)
Limitations and considerations
- Since the ReflectionCaptures capture the scene from a single point and project it from there, it requires precise placement (just as it is for reflections, but here it's much more evident)
- Areas with a clear lighting directionality (i.e. bright spots on one side of a room) require multiple ReflectionCaptures to avoid the light getting projected unnaturally
- As IBL overrules skylighting (intended), anything outdoors that's affected by reflection captures won't be affected by skylighting. Therefore it's recommended to only use it in fully occluded areas (interiors) or semi-occluded areas (i.e. a forest) but not on clear outdoor areas
- On areas with "sharp" indoor/outdoor delimitations (i.e. building walls) it requires very precise placement to avoid indoor <--> outdoor leaking (not just keeping the skylight from coming in, but also avoiding darkening the outer walls), so thick walls are needed
- On areas without "sharp" indoor/outdoor delimitations the capture radius/size can be kept much bigger for a smoother transition
- As the engine is limited to 341 ReflectionCaptures there's a tradeoff between precision and level size (it's probably not feasable for really large levels that require precise placement like a city with building interiors)
- Translucent materials only work with local IBL when using the 'Surface' Lighting modes. The 'Volumetric' Lighting modes only ignore skylight altogether and do not make use of the reflection captures. You can get the old behavior by commenting out lines 562 and 567 on BasePassPixelShader.usf, but then they will again be affected by the skylight (and they will still not be affected by the reflection captures).
- If using the Forward Renderer, it requires to recompile all shaders (the toggle isn't instant as with deferred).
Additional Technical details
- The gist of it: Reflection Captures are passed over as IBL in the form of ambient light using the highest mip of the reflection.
- Localized IBL is applied in the same shader as the light from the SkyLight, but areas affected by IBL are no longer affected by the SkyLight (smoothly blended). This effectively allows using Reflection Captures as "skylight blocking areas" even if ambient light from Localized IBL isn't wanted.
- The new ContributionFactor property on ReflectionCaptures scale the contribution of that ReflectionCapture (both into ambient light as IBL and into the reflections)
- Dynamic lights are captured into IBL based on the existing 'Indirect Lighting Intensity' property of each light, which can be set to 0 to make it not contribute any light to IBL. This is useful to skip capturing lights that are expected to change during gameplay.
- The entire thing is toggleable via a new CVar "r.LocalIBLFromCaptures" and when disabled the code/shader footprint is next to nothing
- DFAO is applied also in IBL-affected areas, inherits all DFAO properties from the SkyLight
- All of the IBL light contribution will be affected globally with the PostProcessVolume's 'Indirect Lighting Intensity' and 'Indirect Lighting Color' properties
- Works only when using a Movable Skylight - gets properly disabled on Static or Stationary SkyLight - works even alongside a baked Static or Stationary DirectionalLight (but who would ever want that)
- Works with both Sphere and Box Reflection Captures
- Works using Reflection Captures' Source Type as 'Captured Scene' and 'Specified Cubemap' as well
The engine changes are relatively small (under 200 lines of code between C++ and HLSL) and the footprint when the feature is disabled is next to nothing. The pictures speak for themselves and local ambient light (or at least a way to block the skylight) is something we've been asking for a long time
Also I'd love to see this in actual game scenery so if someone integrates this into a custom build don't hesitate to post comparison screenshots

Comment