Coming over from the SVOGI thread [COLOR=#252C2F]and the Physical Lights thread [COLOR=#252C2F]I decided to make a new thread and see if there’s any interest on this.
[COLOR=#252C2F]The point is an attempt to repurpose the Reflection Captures into Localized Image-Based Lighting[COLOR=#007000][COLOR=#007000].[/COLOR][/COLOR][/COLOR][/COLOR][/COLOR]
The expected result would be a setup of baked ambient lighting (without using Lightmass) that goes beyond the limits of a globally-affecting skylight, that can co-exist in a dynamic lighting setup.
The most important use case would be indoor-outdoor ambient light differentiation, such as going inside a house or cave and not having the skylight affecting it. This is a commonly requested feature that is so far not addressed.
Up to UE4 4.11 there was a hidden feature called DiffuseFromCaptures which apparently was a way to contribute to lighting via reflection captures but it felt very short at it[COLOR=#252C2F], and was removed anyway.[/COLOR]
So I started a proof of concept which turned into a feature
What this does TL;DR - Reflection Captures (Box & Sphere) are repurposed and used as Image-Based Lighting probes, which provide local ambient light while “blocking out” the skylight - allowing proper interior lighting when using a dynamic skylight.
Some comparison shots:
How it looks if you smoothly change the ReflectionCapture’s contribution at runtime:
Test on a production-like environment: UT4’s awesome Outpost23 map
Outpost23 comparison screenshots here: Imgur: The magic of the Internet
And the classic Sponza Atrium scene
How to set up The engine changes were meant to go into the engine as a Pull Request, -integrated and maintained by Epic- and available for everyone directly from the binary engine. Because who wouldn’t like having an additional ambient lighting choice when working with dynamic lights?
Unfortunately the Pull Request was declined by Epic, pretty much coming down to this:
While I understand this system is not a great fit for the engine’s straightforward “Static / Stationary / Dynamic” mobility concept and the grasp of light-object interaction it produces, there are a number of things that don’t fit that view either (i.e. translucency in general, screen-space reflections).
Moving forward the system is available to integrate manually, for which you’ll need a custom UE4 build.
The code is available on my GitHub Fork, where I add branches corresponding to the major engine versions (ignore the ‘master’ and ‘release’ branches).
The expected workflow is to compare my fork with the original UE4 codebase, and then manually integrate the highlighted differences into your own build. I only update my branches to major engine versions, so if you use my fork directly you’ll miss out on all engine hotfixes.
Supported versions are 4.20, 4.21, 4.22, 4.23, 4.24, 4.25 and 4.26.
I don’t actively use the system and maintaining custom engine versions requires a too big amount of time.
With the announcement of UE5 and its Lumen system for GI I have decided to stop maintaining this system. I will support it until the last version of UE4 (4.26 or 4.27), but will not carry it over to UE5. Therefore if you are thinking about newly integrating this system I’d recommend you to look for other approaches.
For dynamic lighting and handling of indoor/outdoor ambient lighting here’s some alternatives that might be suitable depending on your projects’ needs:
- Use a stationary skylight, and precompute the lighting for it. this way the ambient light will be local per area. you can still have a movable sun light if you need dynamic time of day. if your skylight is stationary you can animate the skylight color with your time of day.
yes, this means using lightmass and baking lighting. however if you force every actor’s mesh in your scene to use the Volumetric Lightmap, you don’t need to create lightmap UVs at all.
If you need an immediate solution personally this would be my preferred method. as long as you don’t need to spawn static geometry at game time (i.e. having procedural levels) I find this to be a very good compromise between quality (much better than a movable skylight) and iteration time (you bake lighting yes, but no need to worry about lightmap UVs)
- If you really need a movable skylight it’s often suggested a ‘solution’ with some sort of custom volume, sort of like a tunnel, which modifies the skylight intensity as you pass through it. this way when you enter the cave it gets darker as you pass the entrance, and find it to be dark when you’re finally inside. I find this solution to be very lacking because if you go inside and get darker skylight and look outside it will be very dark. and if you go outside and look inside it will be very bright. so you need to come up with more and more trickery to avoid this effect (like putting a dark semi-transparent plane to darken the inside, using postprocess volumes, etc)
- Try using Distance Field AO. it works well up to a certain size, but once your interiors are big enough the occlusion becomes too “local”, and the floor/walls/ceiling are too far apart to occlude eachother and you’re back to having the unwanted outdoors-like ambient light inside.
- Use a very dim skylight (tuned for interiors) and couple it with LPV. the idea is to tune the LPV values to be strong enough for the GI to act as ambient light. I’ve seen it work well enough, it’s just limited by the LPV featureset (very coarse voxels causing bleeding on thin walls, limited GI distance with a hard cut, performance not as good)
- Use raytracing in your skylight, but this limits your game to users with RTX cards.
- Wait for UE5 to come out next year and use the Lumen GI lighting system.
- Use AO decals as a trick to block the light from a Movable Skylight: make a decal, make a decal material for it, set the Decal Blend Mode to Ambient Occclusion, set the material AO output to 0, fill your interiors with these decals, profit.
it’s probably drawcall heavy and requires a lot of placement. but you can effectively block all dynamic skylight with it. the decals can fade out by distance so you can fine tune your performance vs distance tradeoff, and since it’s material-based you can do gradients for transitions.
How to use
- set r.LocalIBLFromCaptures to 1 via console (set to 0 to disable again if needed). you probably want to add this into the ConsoleVariables ini
- You’ll need to re-compile all Translucent materials manually, since the flag is not propagated for those until the shader is compiled.
- Make sure you have a Skylight that is set to Movable
- Place ReflectionCapture Actors (sphere or box) encompassing the areas where you want Local IBL
- Build the Reflection Captures to get instant results
- Tweak your lights’ “Indirect Lighting Intensity” to control how much each light contributes to the Local IBL (you probably should set it to 0.0 if the light will change during runtime). Build the Reflection Captures when finished
- Optionally tweak your PostProcessVolume’s ‘Indirect Lighting Intensity’ and ‘Indirect Lighting Color’ (applies as global changes) if needed. Build the Reflection Captures when finished
- Optionally modify each ReflectionCapture’s ContributionFactor value in cases where the full IBL isn’t wanted (both at editor time and at runtime)
Limitations and considerations
- Since the ReflectionCaptures capture the scene from a single point and project it from there, it requires precise placement (just as it is for reflections, but here it’s much more evident)
- Areas with a clear lighting directionality (i.e. bright spots on one side of a room) require multiple ReflectionCaptures to avoid the light getting projected unnaturally
- As IBL overrules skylighting (intended), anything outdoors that’s affected by reflection captures won’t be affected by skylighting. Therefore it’s recommended to only use it in fully occluded areas (interiors) or semi-occluded areas (i.e. a forest) but not on clear outdoor areas
- On areas with “sharp” indoor/outdoor delimitations (i.e. building walls) it requires very precise placement to avoid indoor <–> outdoor leaking (not just keeping the skylight from coming in, but also avoiding darkening the outer walls), so thick walls are needed
- On areas without “sharp” indoor/outdoor delimitations the capture radius/size can be kept much bigger for a smoother transition
- As the engine is limited to 341 ReflectionCaptures there’s a tradeoff between precision and level size (it’s probably not feasable for really large levels that require precise placement like a city with building interiors)
- Translucent materials only work with local IBL when using the ‘Surface’ Lighting modes. The ‘Volumetric’ Lighting modes only ignore skylight altogether and do not make use of the reflection captures. You can get the old behavior by commenting out lines 562 and 567 on BasePassPixelShader.usf, but then they will again be affected by the skylight (and they will still not be affected by the reflection captures).
- If using the Forward Renderer, it requires to recompile all shaders (the toggle isn’t instant as with deferred).
Additional Technical details
- The gist of it: Reflection Captures are passed over as IBL in the form of ambient light using the highest mip of the reflection.
- Localized IBL is applied in the same shader as the light from the SkyLight, but areas affected by IBL are no longer affected by the SkyLight (smoothly blended). This effectively allows using Reflection Captures as “skylight blocking areas” even if ambient light from Localized IBL isn’t wanted.
- The new ContributionFactor property on ReflectionCaptures scale the contribution of that ReflectionCapture (both into ambient light as IBL and into the reflections)
- Dynamic lights are captured into IBL based on the existing ‘Indirect Lighting Intensity’ property of each light, which can be set to 0 to make it not contribute any light to IBL. This is useful to skip capturing lights that are expected to change during gameplay.
- The entire thing is toggleable via a new CVar “r.LocalIBLFromCaptures” and when disabled the code/shader footprint is next to nothing
- DFAO is applied also in IBL-affected areas, inherits all DFAO properties from the SkyLight
- All of the IBL light contribution will be affected globally with the PostProcessVolume’s ‘Indirect Lighting Intensity’ and ‘Indirect Lighting Color’ properties
- Works only when using a Movable Skylight - gets properly disabled on Static or Stationary SkyLight - works even alongside a baked Static or Stationary DirectionalLight (but who would ever want that)
- Works with both Sphere and Box Reflection Captures
- Works using Reflection Captures’ Source Type as ‘Captured Scene’ and ‘Specified Cubemap’ as well
The engine changes are relatively small (under 200 lines of code between C++ and HLSL) and the footprint when the feature is disabled is next to nothing. The pictures speak for themselves and local ambient light (or at least a way to block the skylight) is something we’ve been asking for a long time
Also I’d love to see this in actual game scenery so if someone integrates this into a custom build don’t hesitate to post comparison screenshots