BaseLightmass.ini - A Summary From Various Posts

[tupikp]

Q & A Session With DanielW

Here’s the Q & A Session With DanielW

@DanielW;406009

In the meantime, here’s an overview of the methods used along with their pitfalls. I tried to inline the relevant BaseLightmass.ini settings. First, there are two completely separate paths for punctual lights (directional, spot, point) than for sky lights.

Lightmass is heavily optimized around solving indirect lighting from punctual lights with high quality. In particular, the case where you have bright sunlight coming in a small window and hitting the wall/floor, lighting up the whole room, is what I would consider the primary use case. Lighting with sky lights / HDRI is a much more recent implementation and honestly it’s not very high quality.

Punctual light method:

The short summary is that Final gathering is used to solve the first lighting bounce, and photons are used for bounces after that along with guiding the final gather.

1. Many photons are emitted from the light and deposited on the scene as directly visible (direct photons). We bounce these once and record any direct photon paths that resulted in a bounced photon being deposited. This identifies small (but critical) light entrances like windows. DirectPhotonDensity controls how many photons are emitted in this stage, and IndirectPhotonPathDensity controls how many paths are recorded (how accurately we find small windows).
2. Many more photons are emitted from the light, along the direct photon paths. These are bounced for as many times as you requested NumIndirectLightingBounces. Each bounce has a much smaller number of photons successfully bouncing so more bounces after 2 has only a small impact on build times. IndirectPhotonDensity + IndirectIrradiancePhotonDensity control how many photons are emitted to represent multi-bounce lighting.

Now that the photons are ready for the entire scene we can start operating on each lightmap texel.

3. Direct shadowing is computed for static and stationary lights. For static lights which use area shadows, NumShadowRays and NumPenumbraShadowRays control the quality of the penumbra, more samples are needed to support very large, smooth penumbras.
4. We check to see if this texel is covered by other irradiance computations (steps 5-7). If there is, we reuse these nearby irradiance samples through interpolation. If not, we continue to steps 5-7. This is called Irradiance caching. Too much interpolation causes indirect shadows to be lost. The amount of reuse is controlled by RecordRadiusScale, PointBehindRecordMaxAngle, InterpolationMaxAngle. During the interpolation, we can gather even more nearby irradiance samples to further smooth the lighting.
5. Nearby indirect photons are gathered around the lightmap texel. These tell us where most of the bright lighting is coming from, eg the small bright spot that lights up the entire room.
6. We begin the final gather. Rays are traced out in all directions for the hemisphere of the texel. At the end of the ray we either hit some geometry, where we gather the lighting from photons at that point, or we miss geometry and hit the sky, where we evaluate the sky lighting. NumHemisphereSamples controls how many rays there are in this first refinement step.
7. We do multiple iterations of refining the final gather. Each sector of the hemisphere is subdivided and we trace more rays to discover incoming lighting. We subdivide deeply around incoming indirect photons to make sure we resolve the small bright spot on the ground lighting the whole room. This is a form of importance sampling, also called importance driven final gathering. We also subdivide anytime there are significant differences between neighbors to reduce noise, this is called adaptive sampling. NumAdaptiveRefinementLevels controls how many refinement levels there are, AdaptiveBrightnessThreshold controls how big of a neighbor difference there must be to refine a sector.

As to the Lightmass WorldSettings options, IndirectLightingSmoothness operates on the settings in step 4), the amount of irradiance cache smoothing.
IndirectLightingQuality increases the number of samples and depth of refinement in steps 5-7, the final gathering.

Sky light method (assuming static skylight, stationary is a bit different):

1. The sky light HDR texture is exported to Lightmass as an 3 band spherical harmonic, which you can think of as an extremely low resolution cubemap (2x2 on each face approx). So if you have a very bright spot in the sky cubemap it will be spread out over a large region. This is basically prefiltering to avoid noise.
2. We do a mini final gather at each lightmap texel and compute sky lighting with a very low number of rays. This is cached on the surface and will become the first bounce of skylight GI once the final gather accesses it.
3. During the final gather, any rays that hit the scene look up the cached sky direct lighting from step 2), which now becomes first bounce sky lighting. Any rays that miss the scene and hit the background lookup from the sky SH, this is sky direct lighting. We refine the final gather based on brightness differences between sectors of the hemisphere, but no importance sampling is done because we don’t have any photons from the sky.

So now you can see why static sky lights have limited quality and only support one indirect lighting bounce. This is something we could improve, but it’s a lot of work and complexity. Specifically, we should actually export a cubemap to Lightmass, and do importance sampling of the cubemap’s contents (trace more rays to the brighter texels).

@DanielW;410960

Those leaking artifacts look like insufficient photons. Try jacking up DirectPhotonDensity, IndirectPhotonPathDensity, IndirectPhotonDensity, IndirectIrradiancePhotonDensity in BaseLightmass.ini by a factor of 10 or so.

I am still looking into the Skylight-only case. I found a bug in the adaptive sampling, such that it was not doing much with IndirectLightingQuality > 1. Also I implemented portals and I’m testing those out. These things take time though, it will be some weeks before I have good results - mostly as I have to make sure no quality or build time regressions occur in many maps.

IndirectPhotonEmitConeAngle is used in combination with IndirectPhotonPathDensity to channel indirect photons into small windows and doors where they are needed the most. In a scene like that (indirect lighting of local lights not doing much) I don’t know why it would matter.

@DanielW;411223

Also, does uncompressed lightmaps makes a noticeable difference?

It’s very noticeable to me esp if you look at a shadow penumbra that should otherwise be smooth.

@DanielW;412453

@Nicolas3D : What I have observed is, if we are using other lights along with Skylight we don’t need the high Photon Density(Direct as well as Indirect).

Skylights shouldn’t interfere with other light’s indirect lighting. So if you need to jack up photon density for some reason, introducing a skylight will not change that.

Increasing the Photon Density, increases the build time by the factor of amount of increase from default value(400).

Increasing photon density will only increase the photon emission part of the build by that factor. Photon emitting is usually 15% of a build, so you can raise it quite a bit before doubling your build time. Look at the stages in the swarm visualizer or the swarm log stats once a build completes. In contrast, NumHemisphereSamples and StaticLightingLevelScale are probably the quickest way to increase your build time.

@DanielW: Are portals going to be included in a release soon?

Well they still only exist on my computer so… Realistically it will be 4 months before any code change I make today will make it into a released build. Of course you can build latest directly from github (once it’s checked in), but that tends to be very unstable.

@DanielW;412458

One major improvement I’d like to see to lightmass is an improvement to the map processing. Currently it’s not possible to divide a single texture up across multiple threads, so build times are always capped at the largest lightmap resolution you have in the scene (usually anyway), and only one thread can do that work while the rest are doing nothing.

That was true in UE3 lightmass, but it’s been improved in UE4 lightmass. Specifically, the final gather is split into tasks of 128^2 and distributed to any threads that are idle. The largest lightmap component will still be the bottleneck, but your CPU is better utilized during it. This makes the biggest difference in small scenes where you are going for high quality, so there aren’t enough components to keep all threads busy. This is done in FStaticLightingSystem::CalculateIndirectLightingTextureMapping in code.

Now, there are still other parts of lightmass work for a simple component’s lightmap that could be multithreaded better - specifically area shadows.

…or ability to choose number of cores, so one can surf/whatever while lightmass is calculating… just a thought…

This can be set in Swarm agent settings. You can bring up the swarm agent by starting a lighting build once (in a tiny map, or just cancel it), then find the icon hidden in your taskbar and double click it.

@DanielW;412464

Fantastic results from the portal lights.

Portal light is a bit of a misnomer, they’re not lights as they do not emit light. ‘Light portal’ makes sense though.

Which shouldn’t be a problem with 12gb vram of Titan-X, but I couldn’t stop this process by changing all .ini files I found on internet. No matter if I set shadow/lightmap mip-maping to 0/false, or rised value for memory pools, both in project and general folders, etc… it would always mip-map those lightmaps,
which is just visually horrible. Anyone with similar issue got this solved ?

Make sure you are on Epic scalability settings.

Texture streaming poolsize settings come from Engine/Config/DefaultScalability.ini

[TextureQuality@3]
r.Streaming.PoolSize=1000

You can also disable lightmap streaming in your project’s DefaultEngine.ini. You have to rebuild lighting and resave after changing this. Here’s the default (from Engine/Config/BaseEngine.ini)

[TextureStreaming]
AllowStreamingLightmaps=True

@DanielW;413472

I have now checked in my Lightmass solver quality improvements:
A) Adds light portals - these solve static skylight quality in highly enclosed areas like the example
B) Fixed high IndirectLightingQuality settings disabling adaptive sampling - this improves static skylight quality
C) Fixed light leaks when using spotlights with StaticLightingLevelScale .1 - StaticLightingLevelScale is no longer applied to photon emission

These changes will be in 4.11 which will probably be in about 4 months.

So to wrap up Raghu’s original post

1. How to remove the splotches in the area where there is no skylight hitting directly (Areas far from windows). As you can see in the above images there are so many splotches in gallery, despite having Indirect Lighting quality to = 10 and static Lighting Level scale = .1
2. Shadows become somewhat messy after a certain distance. How to make area shadows more clear and smooth.

This is addressed by A) + B)

3. Whenever we put spotlights to remove dark areas, it almost always gives the annoying problem of light leaks. Clearly evident in the above images.

This is addressed by C)

Sadly that’s all the time I can spend on Lightmass atm, otherwise I would like to investigate some more complex scenes.

@DanielW;414135

Is there a workaround for point C) at the moment?

If possible kindly provide us the solution to the - “Light leaks due to spot/point/direct lights in static mode” as I can safely assume that you have already figured it out.

The light leaks that occur with a stationary spot light come from application of the .1 StaticLightingLevelScale to photon settings. In particular, a ton more photons are emitted at StaticLightingLevelScale .1 and the search distance is decreased. This is supposed to increase the quality of the photon representation, but for some reason (I have not been able to determine) it results in a huge amount of variance / noise / splotchiness. This causes the phantom lights ‘light leaks’ in your images.

I simply disabled application of StaticLightingLevelScale to the photon settings in code. You can probably achieve the same workaround by increasing IndirectPhotonSearchDistance by 10x if you use StaticLightingLevelScale .1, effectively canceling out the scale. I have not tested this myself though.

@ DanielW - As now I am trying to learn Lightmass and Photon mapping deeply, It’s kind of a mystery to me, what these settings change (in BaseLightmass.ini or may be in some other config files) when we manipulate them in editor -

Static Lighting Level scale from 1 to .1

Indirect Lighting quality from 1 to 10

Indirect Lighting Smoothness from 1 to .25

StaticLightingLevelScale - this scales the density of every computation lightmass makes. It has two big noticeable impacts - for final gathering (which computes first bounce GI from lights + sky light direct) it increases the density of light gathers. It won’t actually increase the quality of light gathers, just how far apart they are on a surface. This improves contact shadows but explodes your build time. It also increases the density of photons emitted, this is what I disabled in my latest change to work around the phantom lights.

IndirectLightingQuality - this mostly increases the number of rays traced at each final gather, also known as NumHemisphereSamples in the ini. A value of 10 means 10x more rays and therefore 10x longer build time. You can think of more rays as a higher resolution image being resolved, so it improves the quality of light gathers.

IndirectLightingSmoothness - we filter between light gathers a bit to hide the last little bit of noise. The default amount of smoothing is quite high, so it can be useful to lower this to get more detailed shadows. Note that smaller values for StaticLightingLevelScale also decrease smoothness, so with a StaticLightingLevelScale of .1 you probably don’t need to reduce smoothing anymore. IndirectLightingSmoothness doesn’t affect build time.