How does one build lighting for massive worlds?

Assuming I have a bunch of streamed levels, huge terrain from World Machine… how does light building work in that scenario? My stupidly powerful desktop machine is crying when I try to do it on a 2 x 2 kilometer terrain. I fret to think what would happen on our final world size. I understand that the swarm agent + a bunch of studio PCs will be involved, but what if, after an all-night build, one tiny dot is wrong. Does everything need to be redone from scratch? What are some best practices in this field?

I’ve wondered about this also. Specially the question about recomputing everything if there is an annoying little bleed somewhere…

Hi DamirH,

I’ll speak about it from a high-level rather than the exact specifics.

Doing a large world is better to break your terrain up into tiles and use with level streaming and World Composition. This is how large worlds are usually handled. Having lightmass bake for significantly large terrains will require a lot of RAM or lightmass may crash in the process of building. Dividing the world into smaller tiles and baking those independently using the levels window can be more beneficial if you run into a situation like you mention and need to rebuild for a smaller area only.

The more objects that you have in your scene that need to be baked with lightmass will require more RAM. To Look at an example that’s provided on the Marketplace. The Landscape Mountain Demo takes ~24 GB of ram to build lighting using lightmass.

Key things to remember is to use world composition, level streaming, include a lightmass importance volume in your level that covers playable areas (You can have multiple of these), and keep lightmap resolutions within reason (Do not make everything have large Lightmap resolutions. for most objects 32-128 will suffice).

World Composition Docs: World Composition in Unreal Engine | Unreal Engine 5.1 Documentation

Level Streaming Docs: Level Streaming in Unreal Engine | Unreal Engine 5.1 Documentation

I hope this helps. :slight_smile:

Tim

2 Likes

Tim - I’m sorry I wasn’t clear enough. I already use world composition and my 2x2 kilometer terrain is split into 16 tiles. I am not sure if I can bake them separately? What if a mesh from tile A casts a shadow into tile B? Can I bake them in chunks (will it add up or replace the previous light maps?).

Either way, I believe that lightmass isn’t the right solution for large worlds due to the size of the lightmaps. I am going with LPVs for now and am hoping that the new dynamic GI solutions will hit the engine soon.

this is exactly what I was wondering. I have one I’m working on that is just under 500 tiles. Luckily I think I can do a few loaded at a time and group them so that is there abnormalities they can be areas that don’t really get notice.

Something else I noticed is that in a single terrain can also have its individual light map resolution overridden. turning down tiles that fall outside view can really help. Also individual sections of terrains can be deleted as well. like I had a back side of a mountain range there was a big chuck of the terrain that was back there that would never be seen so just selected the components and deleted them.

As I said earlier, LPVs seem to be the way to go for this. Lightmaps take up way too much memory for this scale.

Lightmaps for large worlds would be usable only if engine could bake selected objects/scenes instead of whole world at a single shot.
My background scene has a bazillion props and I didn’t even try to bake lighting for it, I lightmap only playable areas.

Sorry for the delay in reply. It’s been a busy couple of days! :slight_smile:

Well I can say you’re off to a good start by already using world composition! Your concern about what’s baking across tiles is a valid concern and there are some implementations coming to better handle large world support. I’m not sure what all this entails with lightmass and baking or any of the specifics. Typically features that may be included in our internal build do not have documentation and feature list fully completed until close to the release of each engine version so I can’t say for certain how this would help yet.

Large worlds are a hard area to help with lightmass simply because most users are not using a build farm or multiple computers to build lighting. Using a single computer has it’s limitations for sure as you can’t let swarm agent have other machines build parts of the level. This can often lead to lightmass crashing because it runs out of memory or taking a long long time for it build.

You can bake lighting by having the levels visible that you want to build lighting for. Anything not visible will not be built. When lighting is built for these it will replace the previous lightmap of the level that was visible. Making a level not visible and building another level will not erase the previous levels lightmap just because it was not visible.

A lot of the development community have found LPVs to be useful but there are some draw backs and light leaking with this as well. This is still an experimental feature. I’d say, from what I’ve seen others post, that if your world is mostly out doors and minimal interiors (unless you’re loading in different interior levels) that you will run into the light leaking that others have reported.

Personally I’ve used Distance Field Soft Shadows more. I like the look of the dynamic shadows using this method vs cascaded shadow maps, although it comes with its own limitations at the moment since it’s only been released for a couple of versions. Daniel has even worked on getting Distance field global illumination in the engine which you can see first hand here: Global Illumination alternatives - Feedback & Requests - Epic Developer Community Forums

To wrap it all up, I still think lightmass and baking lightmaps is the way to go because you cannot deny the benefit of not having to dynamically cast a shadow vs a bake shadow to save performance. Since most large world games are developed by bigger game studios they would use a build farm for their light building vs a single machine or even a couple.

Tim

Hey Tim,

Thanks for the detailed response! The “baking each level separately” information is great, that’ll help us a lot. My world is 90% outdoors at this prototype stage, and I’m experimenting with various approaches (the light is on the backburner for now, hashing out the actual gameplay). I am hopeful that by the time we enter full blown production, Distance Field GI will be stable enough to use.

Best regards,
Damir H.

Lighting is definitely something that should be thought about in the process. Making a decision early on with some testing is a good idea, especially if you’re looking to go with baked lighting. Lightmaps would be a huge undertaking at the end of your project if you hadn’t already set them up. Dynamic lighting is much easier in that you don’t have to worry about the lightmaps, but there would be other performance refinements to keep things more optimized (ie. disabling dynamic shadows on objects that dont need it, tweaking shadow quality, using profiling to find the hiccups).

Daniel is making some great strides with Distance Fields and I’m always excited to see what he’s doing next! Those are by far some of my favorite updates with the engine.

Tim

I know this is a silly thing to ask but could you poke him about an ETA or an update related to it? Will we be seeing it in 4.8 or dare say 4.7?

It’s probably best to follow this thread: Global Illumination alternatives - Feedback & Requests - Epic Developer Community Forums

The start of the distant field GI is in CL: 2390645 or later (which you can grab off GitHub and build from source.

4.7’s CL for the preview is 2388154 so it most likely won’t be included before 4.8’s release via the launcher.

Like he says, this is the first of 10 steps to get where he wants. He is continually making improvements. I suppose that development for this will be much like it has been for DFAO and DFSS. There are improvements that come with each engine release. 4.7 will improve some of the issues, but not all, with DFAO. Keep an eye on the release notes as this will have the most up to date information for each release.

Tim

I`m a little bit confused, so decided to re-ask some things. So for open world it is better to use Dynamic lighting, as I understand it is in raw stage and image will be not that good as it could be baked with usual way. If I need to bake lightmap for specific object, for example static mesh of house, I should disable other meshes except of terrain and that house? Then it will be baked very fast? Thanks in advance

I can say out of experience that Lightmass makes absolutely no sense for large open worlds! We tried it as well and there are just too many limitations. Even if you can bake everything (which will be almost impossible in a huge world with tons of objects) you will run into texture streaming issues at one point.

To put this into some context…we, at our company, havent had any success with lightmass for DI2. And we have quite some resources available :wink:
So we went for a dynamic setup (which btw most of the released open world games do as well) and started tweaking things in the engine to better support this.

Right now, Unreal does not offer a full system for games like that out of the box. If you have some graphic programmers at hand, you can develop and integrate your own solution to make it work. So until there are new features, everybody has to hack around by themselves to find something that works in their environment.

Thats the best help I can give at this point.

Cheers!

Daedalus51 thank you for this important share of experience, so the only one solution is Dynamic lighting which is in raw stage from Epic games, I have heard that Epic are planning to use Enlighten engine, but affraid it will be high prize engine for us as a small team. I have tried to enable Light Propagation Volume in .ini files and using it, the amazing thing about it is, that you don’t need to make lightmap channels and other important things before bake, however images is too bright and needs some tweaks as you said.

yeah…it also doesnt really support a proper reflection environment and because of that, everything metal looks like ****^^ As I said, the biggest problem is just that there is no “full” dynamic replacement for all of the feature sets lightmass provides. You can do this, but then it lacks that, if you do that, it lacks this. The only thing you can do is to decide whats most important to have and then live with the lack of other features.

BTW, I also worked with an Enlighten Integration in Unreal and I dont know if that has changed, but back then, we had huge troubles with Enlighten and large landscapes. And Enlighten needs prebaking as well…so not really less time consuming than lightmass after all^^ Just dynamic light moving in runtime with updating gi.

Cheers!

Well I wouldn’t mind the pre-baking in Enlighten, but it’d need to be able to take advantage of UE4’s swarm agent (or maybe it has its own job distribution solution?) before being viable for large worlds. I believe that, for a fantasy style game set mostly outdoors, LPVs will do just fine.

Enlighten baking can be distributed via IncrediBuild :wink:

Sounds great then. Just need a non-6 figure price tag now and it’s good to go :stuck_out_tongue:

Apologies for hijacking this thread.

We’re in a similar situation to those described here, but with each level in the composition being a distinct sector that is offset in the persistent-level to avoid overlap of playable spaces. Disabling levels to bake works for light-maps, but removes volume-lighting samples for any level that’s disabled, meaning we only have one sector with valid lighting-cache data at any time.

Should we assume that the options available right now are LPVs or no IDLC? That’s fine if so, but it seemed worth asking :smiley:

We were previously baking lighting in the sub-levels themselves, but the volume lighting samples aren’t offset with the level when we move it in the persistent-level. Is there any way of offsetting volume lighting samples, or is that pretty involved?

Thanks!