Static/Stationary/movable openworld!

Development Rendering
1

Hiya.
Still a new folk to UE4, so trying a few things today that got me a bit confused, so im hoping you guys can help clear this up for me.

Im doing an open-world (city) style game so there is lots and lots of buildings! - each of these are there own asset… right.

So now comes the topic of lighting. - Ive been thinking about the best approach for this, but again not knowing the lighting system that great yet Im hoping you can shoot me down if Im a little lost.

Due to the size of the game map (city) its fairly large, so we are using a very aggressive LOD system. with over 500 Buildings.

Option 1

-Thinking to go one single Directional light (sun)
-HDRI On the skylight so we have consistency in uniform lighting across the map
-This will save memory as it wont generate Lightmaps?
-Really use the AO to bring out some of the details.
-We can then use Spot lights/point lights for anything fancy.
-All the above will be set to “movable”

negatives? - No GI, or scatter light. can be seen to be “boring, or Flat” - might cause issues later? no true night/day scenes (as HDRI sucks for night-scenes, unless swapped)

Option 2

-Sunlight/Skylight (all lights) set (STATIC or STATIONARY?)
-To generate light-maps for everything.
-We can then have GI too?
-point-lights are static too, as there are no “dynamic” operations with lights so to speak…
-problems are memory? - and or being forced to use really low light-maps (32 or 64)

negatives? - bad AO/Blocky, and memory

So with that said. - Im looking at these options, so want to fact-find before diving right in

my main questions are, of course, we want the best look vs performance :slight_smile: - (guess you never heard that before!) - its a realistic(ish) game (Think Tom Clancy The Division)

Am i right in thinking for an open-world game on a big scale such as this, light-maps is a NO-GO? - and it will have to all be Dynamic (set to movable lights)

or is there a trick im missing, not knowing about or just being plain daft!.

AS said Im new to the Ue4 workflow. so forgive me if im totally wrong on all the above.
But please assist where you can, i would much appreciate it.

Thanks

2

Hi, you might wanna read this thread here https://forums.unrealengine.com/development-discussion/rendering/26518-how-does-one-build-lighting-for-massive-worlds

I never tried building lightning for a really big world, but since you got streaming/mip mapping for the lightmaps and for really big worlds you would use world composition anyway, the memory consumption should be manageable (you can take the biggest world you want to build for your game, dump a huge amount of meshes in it and see how much memory the lightmaps use in game, so sort of worst case test of what your final map would be). Of course the light will be static then, no dynamic day/night cycle and you might have very long light building times.

Yes, due to several light bounces. All the lightning information gets precomputed and put into the lightmaps. The static lights do not exist in game after light build (that’s why you see those spheres in the level, that’s where the lightning information at this point is saved to light movable objects). And yes the ambient occlusion might require a bigger lightmap size to look good.
As for movable lights, if you only use those then you won’t have lightmaps. As for stationary lights, that means you have one static light that gets precomputed, but you also have a movable light. Cause movable lights have a hard shadow and static lights may have a too soft shadow, stationary lights will give you best visual quality at most times, but you will also pay the cost for both static (memory and build times) and movable light (performance).

With movable lights, I would generate mesh distance fields that cast the shadow at further distance (saves performance) and when you use mesh distance fields you can also have ambient occlusion in the sky light.
Since 4.24 there is screen space global illumination for dynamic lightning, but it costs quite much performance and is in beta (you can activate it in the project settings).

You might wanna think about using a mix of movable and static lightning. So a movable directional light with mesh distance fields for outdoors and then put static point lights with low intensity but high attenuation radius for GI, in areas where you need it.

At last, you might wanna watch the videos on unreal engines learning tab on lightning to get a better feeling of how this works in ue4.

3

When you say , Since 4.24 there is screen space global illumination for dynamic lightning, but it costs quite much performance and is in beta (you can activate it in the project settings).,are you saying that instead of on the actual directional light or in addition to it.

I used to have shadows with my directional light,but with sun & sky none at all, any idea if thats a bug or a tic I’ve missed?

Overall happy with lighting atm with new Sun & sky ,but I still get nasty blowouts in many areas of noting but white.One would have thought the new system would have fixed that ?

Using btw, world comp if that matters due to large scene planned.

4

In addition to it, but you see a difference also without a directional light.

Never used the new sun & sky system, so I can’t tell you :slight_smile:

5

Ive been testing various elements. - pretty much as you said. - I duplicated many meshes in the world (not instanced) - and tried various setups. - here are my findings… (right or wrong) - none of this has been done to any precision, and no thought on “looks” just as a tech demo really.

Setup 1.

Using baked lighting, so Static source light, using lightmaps at max 256 quality (average) - baked AO/GI and Some stationary/static pointlightes to fill areas, with skylight to fill up any missing ambience.

Conclusion. - very slow, inconsistency with the quality of lightmaps. and generally quite a flat world (very oldschool look) though this has the best potential to look awesome!. (but requires the most work)

Setup 2.

Using the new Light propagation system, so just a Sun light (no skylight) - and then relying on bounces of real world materials.

Conclusion, this at first at promising results, as everything was truly dynamic, but there is a lot of setup work, and too many variables that could go very wrong, so not really desirable at this stage.

Setup 3.

Dynamic Sun/Sky with point lights representing fake “GI” (where needed) - Used a HDRi as the lightsource, and a good blend between shadows and volumelight.

Conclusion, - this was fast and effective, no lightmaps needed, everything felt balanced. - problems being that to get a good sense of DAYtime light the levels would require boosting. and looked great in some areas, but not so much in others. - so more material work is needed for this to really work well. - although we have been working to a very strict"natural" level in terms of colour, with this system we need to alter that to “fake” colour or to dull them down to work correctly…

So Setup3 is where we are thinking. - getting a good HDRI is the key that is balanced. - and being careful to not get “blownout” as its sooo easy to do with volume lighting in this way. - the sun just is there as a shadow caster but not really adding to the look. - this way we get full SSAO working to our advantage too. - and yup, we have activated distant meshes

I dunno, still as confused now as I ever was haha!. - love to have a mix of all 3 really. - just not a reality.

Not even discovered yet in unreal how to do the whole Day/Night cycle thing, and im hoping our option3 can do this too, though I would suspect YES as thats the point of “movable” lights!.

Btw. I did try the new SSGI thing. but was unable to actually make it work. - and if that is activated then it auto disables SSAO.

Fingers crossed!

P.S - As a side note. - IF we do use option 3 with the choice to use no lightmaps. - im guessing we don’t need a light-mass volume in the scene? or do we still need that anyway?

P.S 2 - If option 1 was to be on the table, then we would generate our own baked AO & Cavity maps outside of Unreal. - though, not sure how to then plug these in to the engine (as all buildings) use various materials, so I cant just plug in into that… - (we also use lots of dedicated Master materials) so any thought on that? how we would get our own Baked Ao and perhaps a bonus Cavity map too?

Thanks as always,
CM

6

I only used light-mass volumes for building static lightning or building precomputed visibility cells, so as far as I know you won’t need it when you don’t have static lightning/ don’t build precomputed visibility.

Hmmm, so if I understand your problem right, then you have the same static mesh but not all those same static meshes have the same material applied to it and you use those materials for different static meshes and your problem is that you don’t want to create material instances for all combinations by hand?

What might work (I never used ue4 editor scripting so I don’t know it’s limitations and therefore can’t tell you whether or not this will work :slight_smile: ):

  • Add an texture parameter for ambient occlusion to your master materials, let’s call it “AO_map_param”.

  • Write a script that creates the material instances for all combinations and applies them to the static meshes.

  • So I would create one folder for all the AO maps, let’s call it “AO_folder” and one folder for all the newly created material instances, let’s call that “Mat_Instances”.
    Then for the script:
    [HR][/HR]

  • I would iterate over all static meshes you have in the content browser/in your level (if you apply different materials to the same static mesh) and use a map/naming convention to map from the static mesh to the AO map and if you haven’t already imported this AO map then import it from your disk into the “AO_folder”.

  • Then iterate over all the materials you have applied to this static mesh (you would again need a map/naming convention to map from static mesh + material to material instance) and if you haven’t already created the new material instance, you would create it inside the “Mat_Instances”, set the “AO_map_param” to the AO map you have imported earlier and apply this to the static mesh instead of the current material
    [HR][/HR]
    I would also write a script to reverse the process, so map from those new created material instances to the old ones and apply those old ones again. So if you would have any problem you could reverse the process and wipe the “AO_folder” and “Mat_Instances” folder clean and run the script above to import and apply them again.

As for the inconsistent lightmap quality, under optimization viewmodes you can switch to lightmap density.
And one thing you also might wanna keep in mind concerning option 1 is the disk space for all your lightmaps. So a 256x256 8 bit lightmap would take 256x256x8x3 around 1.6mbit, and let’s say 500 buildings each consisting of 100 modular parts (I assume you don’t have that many parts but I assume you will also have other things than buildings on your map) that will be
256x256x8x3x500x100 around 9.8 gigabyte disk space. As far as I know ue compresses the lightmaps by default and only saves the compressed ones to disk, so that would result in around 2.5 gigabyte disk space.

ue_lightmap.png
ue_lightmap.png754×93 24.6 KB

7

Hey thanks for getting back once more…

Well you almost had me, but not quite. - I use “unique” meshes. - its a city after-all. a very specific part of the city means no duplication.
What i meant was, On completion of the model, we used to Then go ahead and bake good AO/Cavity maps. - but stopped doing that since we went dynamic and didn’t need them.

but now we are generating SSAO in game, this is far from perfect. and would like a lot more control over AO at least. so we can really “tighten it up” - so this is where our AO/Cavity maps come into play. - the problem is…

We are already using a lot of AO in our PBR Materials (master materials) - and each building has for example 8 materials each. (brick/details/stone…etc etc) - so how would I apply the same “AO Map” to that building

See its a tricky one. and personally dont see how its possible. - You cant add it to the master materials as we do a lot of “resource sharing”. - Ive been wishing now for a “insert external AO map” within the editor on Unreal.- sadly this will never happen (EPIC)

In terms of everything else its going well under the new methods. - apart from the AO thing… but hoping one day… Epic well answer this and have a solution to it!.

Although i did have a problem on load this morning… - last night I had a good “look” was happy with how things were. - this morning, loaded up, and black blobs everywhere. - turned out to be the Distance Field AO. - so I had to turn it off again, such a shame really… cant use that anymore. until its fixed.

Thanks
CM

8

I’m not quite sure whether or not I see the problem… :slight_smile:

If you UV-unwrap your building in a way that no faces overlap (you would also need if you want to switch back to build lightning, and if in your normal workflow they are overlapping you could put this into the second UV-channel) then it won’t matter how many different materials you apply to it, they can all use the same AO map you baked for the building (you would use “TexCoord” with the non overlapping UV-cannel as UV’s for the AO map).

But you would need to create a material instance for each different static mesh (building). You could even do this inside of a construct script of a blueprint, so create dynamic material instance and set texture parameter value for the AO map (so one blueprint that does this on construct for all meshes inside of your level using a map from static mesh to texture 2d). [HR][/HR]
Of course you would need some logic inside the master material to combine the AO map from the whole building with the AO map you have from the texture (brick/details/stone…) is that the problem?

Or is the problem that you can’t put the AO map for the building into the master material cause you use this master material also for things where you don’t want such an AO map for the whole static mesh? If so, isn’t that solved by creating a static switch inside the master material (by default not using the AO map) and then create one instance where you use it and then create further instances from that for the buildings?

EDIT:

You might wanna try moving the camera far away and then back in (that causes to refresh those). Else you might wanna try activating “Two-Sided distance field generation” and see if that may help in that case.

9

Well i fully understand the use and restrictions of Lightmap UVs and how to go about them, thats not the issue. - the concern was we are already using very detailed dedicated master materials. - because of the sheer size, we cant afford to have lots of material elements so clever planning was needed to make the RAM usage manageable. - so we scripted lots of clever elements together that served our purpose perfectly, and gave us multi direction tools that we needed.

We still create a 2nd UV channel, but what I was saying is we not longer bake before bringing into unreal as we used to in the beginning as we was using light-mass

No that we are full dynamic and that baking is no longer on the table. - I simply wanted to find a way to apply our “baked” building Lightmaps to the buildings. - but we DONT want to have a master material per each building as thats a lot of extra drawcalls we cant afford, -we cant add it to the master material as that materiel is then also used on other buildings (instance off)

We will be doing some checking today. - on a small test setup on a smaller prop object to see if baked AO/GI/Cavity etc is best. versus SSAO/GI - as the way i see it, its a lot easier to cull & texture stream than it is for per pixel GPU draw performance. (at least in our previous engine it was like this) - the end of the day, creating a game of this magnitude. small details are key!. and while SSAO is pretty decent, the lack of options for it are not what we need in this case, in our early tech days when we were deciding on engines, we had it all working perfectly, via material instances. - and we did some cool stuff, but since then things have changed somewhat as we need to think about longterm…

The only way I can see this happen is for me to script a new master material, that allows hot-swapping of the various MISC maps I want to use (AO/Cavity) etc. - then apply that new MAP per each material used on the building. as its a instance of of the main mater material this is dooable… but this brings our material count to 200+ (expense)

P.S Thanks for the tip regarding Far Fields. - will give that a go. - I guess they need to fix that! :slight_smile:

10

Ok was back on Distance Fields again today. - good shout on the 2sided thing, - needed to do it per mesh though, so was a bit of a pain. - but it cleared up the horrible mess nicely!.

Although reading the information more, this comes at a lot more cost… - so there must be another way to solve this issues i guess!.

EDIT: - Nope defo cannot do that!. - performance hits a serious nosedive! - so only thing we can do then is disable it and hope they fix it soon, as its not working in any way shape or form at the moment!