Announcement

Collapse
No announcement yet.

How does one build lighting for massive worlds?

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    How does one build lighting for massive worlds?

    Assuming I have a bunch of streamed levels, huge terrain from World Machine... how does light building work in that scenario? My stupidly powerful desktop machine is crying when I try to do it on a 2 x 2 kilometer terrain. I fret to think what would happen on our final world size. I understand that the swarm agent + a bunch of studio PCs will be involved, but what if, after an all-night build, one tiny dot is wrong. Does everything need to be redone from scratch? What are some best practices in this field?

    #2
    I've wondered about this also. Specially the question about recomputing everything if there is an annoying little bleed somewhere...

    Comment


      #3
      Hi DamirH,

      I'll speak about it from a high-level rather than the exact specifics.

      Doing a large world is better to break your terrain up into tiles and use with level streaming and World Composition. This is how large worlds are usually handled. Having lightmass bake for significantly large terrains will require a lot of RAM or lightmass may crash in the process of building. Dividing the world into smaller tiles and baking those independently using the levels window can be more beneficial if you run into a situation like you mention and need to rebuild for a smaller area only.

      The more objects that you have in your scene that need to be baked with lightmass will require more RAM. To Look at an example that's provided on the Marketplace. The Landscape Mountain Demo takes ~24 GB of ram to build lighting using lightmass.

      Key things to remember is to use world composition, level streaming, include a lightmass importance volume in your level that covers playable areas (You can have multiple of these), and keep lightmap resolutions within reason (Do not make everything have large Lightmap resolutions. for most objects 32-128 will suffice).

      World Composition Docs: https://docs.unrealengine.com/latest...ser/index.html

      Level Streaming Docs: https://docs.unrealengine.com/latest...ing/index.html

      I hope this helps.

      Tim
      Tim Hobson | Learning Resources | Epic Games
      UE4 Documentation
      Wiki Troubleshooting Guides: Lighting | Swarm Agent | Destructibles |
      _____________________________________________
      Follow me on Twitter @TimHobsonUE4 | My Personal Website
      Photons Be Free: Mini-Tutorials and other Curiosities

      Comment


        #4
        Tim - I'm sorry I wasn't clear enough. I already use world composition and my 2x2 kilometer terrain is split into 16 tiles. I am not sure if I can bake them separately? What if a mesh from tile A casts a shadow into tile B? Can I bake them in chunks (will it add up or replace the previous light maps?).

        Either way, I believe that lightmass isn't the right solution for large worlds due to the size of the lightmaps. I am going with LPVs for now and am hoping that the new dynamic GI solutions will hit the engine soon.

        Comment


          #5
          Originally posted by DamirH View Post
          Tim - I'm sorry I wasn't clear enough. I already use world composition and my 2x2 kilometer terrain is split into 16 tiles. I am not sure if I can bake them separately? What if a mesh from tile A casts a shadow into tile B? Can I bake them in chunks (will it add up or replace the previous light maps?).

          Either way, I believe that lightmass isn't the right solution for large worlds due to the size of the lightmaps. I am going with LPVs for now and am hoping that the new dynamic GI solutions will hit the engine soon.
          this is exactly what I was wondering. I have one I'm working on that is just under 500 tiles. Luckily I think I can do a few loaded at a time and group them so that is there abnormalities they can be areas that don't really get notice.

          Something else I noticed is that in a single terrain can also have its individual light map resolution overridden. turning down tiles that fall outside view can really help. Also individual sections of terrains can be deleted as well. like I had a back side of a mountain range there was a big chuck of the terrain that was back there that would never be seen so just selected the components and deleted them.
          James Brad Barnette
          Senior Creative Artist
          3Dmotif LLC

          Comment


            #6
            As I said earlier, LPVs seem to be the way to go for this. Lightmaps take up way too much memory for this scale.

            Comment


              #7
              Lightmaps for large worlds would be usable only if engine could bake selected objects/scenes instead of whole world at a single shot.
              My background scene has a bazillion props and I didn't even try to bake lighting for it, I lightmap only playable areas.
              | Finite State Machine | Auto-Save Plugin | USQLite Plugin | Object-Pool | Sound-Occlusion | Anti-Cheat Plugin | Property Transfer Tool | Magic Nodes |

              Comment


                #8
                Originally posted by DamirH View Post
                Tim - I'm sorry I wasn't clear enough. I already use world composition and my 2x2 kilometer terrain is split into 16 tiles. I am not sure if I can bake them separately? What if a mesh from tile A casts a shadow into tile B? Can I bake them in chunks (will it add up or replace the previous light maps?).

                Either way, I believe that lightmass isn't the right solution for large worlds due to the size of the lightmaps. I am going with LPVs for now and am hoping that the new dynamic GI solutions will hit the engine soon.
                Sorry for the delay in reply. It's been a busy couple of days!

                Well I can say you're off to a good start by already using world composition! Your concern about what's baking across tiles is a valid concern and there are some implementations coming to better handle large world support. I'm not sure what all this entails with lightmass and baking or any of the specifics. Typically features that may be included in our internal build do not have documentation and feature list fully completed until close to the release of each engine version so I can't say for certain how this would help yet.

                Large worlds are a hard area to help with lightmass simply because most users are not using a build farm or multiple computers to build lighting. Using a single computer has it's limitations for sure as you can't let swarm agent have other machines build parts of the level. This can often lead to lightmass crashing because it runs out of memory or taking a long long time for it build.

                You can bake lighting by having the levels visible that you want to build lighting for. Anything not visible will not be built. When lighting is built for these it will replace the previous lightmap of the level that was visible. Making a level not visible and building another level will not erase the previous levels lightmap just because it was not visible.

                A lot of the development community have found LPVs to be useful but there are some draw backs and light leaking with this as well. This is still an experimental feature. I'd say, from what I've seen others post, that if your world is mostly out doors and minimal interiors (unless you're loading in different interior levels) that you will run into the light leaking that others have reported.

                Personally I've used Distance Field Soft Shadows more. I like the look of the dynamic shadows using this method vs cascaded shadow maps, although it comes with its own limitations at the moment since it's only been released for a couple of versions. Daniel has even worked on getting Distance field global illumination in the engine which you can see first hand here: https://forums.unrealengine.com/show...l=1#post193884

                To wrap it all up, I still think lightmass and baking lightmaps is the way to go because you cannot deny the benefit of not having to dynamically cast a shadow vs a bake shadow to save performance. Since most large world games are developed by bigger game studios they would use a build farm for their light building vs a single machine or even a couple.

                Tim
                Tim Hobson | Learning Resources | Epic Games
                UE4 Documentation
                Wiki Troubleshooting Guides: Lighting | Swarm Agent | Destructibles |
                _____________________________________________
                Follow me on Twitter @TimHobsonUE4 | My Personal Website
                Photons Be Free: Mini-Tutorials and other Curiosities

                Comment


                  #9
                  Hey Tim,

                  Thanks for the detailed response! The "baking each level separately" information is great, that'll help us a lot. My world is 90% outdoors at this prototype stage, and I'm experimenting with various approaches (the light is on the backburner for now, hashing out the actual gameplay). I am hopeful that by the time we enter full blown production, Distance Field GI will be stable enough to use.

                  Best regards,
                  Damir H.

                  Comment


                    #10
                    Lighting is definitely something that should be thought about in the process. Making a decision early on with some testing is a good idea, especially if you're looking to go with baked lighting. Lightmaps would be a huge undertaking at the end of your project if you hadn't already set them up. Dynamic lighting is much easier in that you don't have to worry about the lightmaps, but there would be other performance refinements to keep things more optimized (ie. disabling dynamic shadows on objects that dont need it, tweaking shadow quality, using profiling to find the hiccups).

                    Daniel is making some great strides with Distance Fields and I'm always excited to see what he's doing next! Those are by far some of my favorite updates with the engine.

                    Tim
                    Tim Hobson | Learning Resources | Epic Games
                    UE4 Documentation
                    Wiki Troubleshooting Guides: Lighting | Swarm Agent | Destructibles |
                    _____________________________________________
                    Follow me on Twitter @TimHobsonUE4 | My Personal Website
                    Photons Be Free: Mini-Tutorials and other Curiosities

                    Comment


                      #11
                      I know this is a silly thing to ask but could you poke him about an ETA or an update related to it? Will we be seeing it in 4.8 or dare say 4.7?

                      Comment


                        #12
                        It's probably best to follow this thread: https://forums.unrealengine.com/show...l=1#post193884

                        The start of the distant field GI is in CL: 2390645 or later (which you can grab off GitHub and build from source.

                        4.7's CL for the preview is 2388154 so it most likely won't be included before 4.8's release via the launcher.

                        Like he says, this is the first of 10 steps to get where he wants. He is continually making improvements. I suppose that development for this will be much like it has been for DFAO and DFSS. There are improvements that come with each engine release. 4.7 will improve some of the issues, but not all, with DFAO. Keep an eye on the release notes as this will have the most up to date information for each release.

                        Tim
                        Tim Hobson | Learning Resources | Epic Games
                        UE4 Documentation
                        Wiki Troubleshooting Guides: Lighting | Swarm Agent | Destructibles |
                        _____________________________________________
                        Follow me on Twitter @TimHobsonUE4 | My Personal Website
                        Photons Be Free: Mini-Tutorials and other Curiosities

                        Comment


                          #13
                          I`m a little bit confused, so decided to re-ask some things. So for open world it is better to use Dynamic lighting, as I understand it is in raw stage and image will be not that good as it could be baked with usual way. If I need to bake lightmap for specific object, for example static mesh of house, I should disable other meshes except of terrain and that house? Then it will be baked very fast? Thanks in advance

                          Comment


                            #14
                            I can say out of experience that Lightmass makes absolutely no sense for large open worlds! We tried it as well and there are just too many limitations. Even if you can bake everything (which will be almost impossible in a huge world with tons of objects) you will run into texture streaming issues at one point.

                            To put this into some context...we, at our company, havent had any success with lightmass for DI2. And we have quite some resources available
                            So we went for a dynamic setup (which btw most of the released open world games do as well) and started tweaking things in the engine to better support this.

                            Right now, Unreal does not offer a full system for games like that out of the box. If you have some graphic programmers at hand, you can develop and integrate your own solution to make it work. So until there are new features, everybody has to hack around by themselves to find something that works in their environment.

                            Thats the best help I can give at this point.

                            Cheers!
                            Check out UNREAL 4 Lighting Academy
                            https://forums.unrealengine.com/show...ng-like-that-)

                            Comment


                              #15
                              Daedalus51 thank you for this important share of experience, so the only one solution is Dynamic lighting which is in raw stage from Epic games, I have heard that Epic are planning to use Enlighten engine, but affraid it will be high prize engine for us as a small team. I have tried to enable Light Propagation Volume in .ini files and using it, the amazing thing about it is, that you don't need to make lightmap channels and other important things before bake, however images is too bright and needs some tweaks as you said.

                              Comment

                              Working...
                              X