Shadow Rendering Strategies and the failing of Level load

Development Rendering
1

The game world I’m working on is quite large. In order for my game to easily save game, I am forced to always bring my game levels into the core of my game, my main Persistent level and turn them on and off.

This sounds logical, but as I approach forty and fifty levels it gets more complicated and the worst part is dealing with rendering shadows. It isn’t enough to turn off let’s say twenty levels and then render only the first twenty levels with shadows, because it will give errors for all the rest instead here is my strategy.

My main persistent level has a core of let’s say ten levels that get loaded and unloaded to keep things moving. I render the shadows.

MyGamePersist
Land
GlobalFX
Lighting
AreaA
AreaB
AreaC
AreaD
AreaE
etc…

So let’s say that above is the basic layout of my game level, and the gameplay is controlled at the MyGamePersist. I render shadows and all is well. On the side I am developing various other levels. A Jump Level let’s say which has it’s own ten levels, a Blue Level, a Lava Level, but each of these may have ten “levels” that get loaded with it.

JumpPersist
Land (the same land just loaded to work out the jump level not rendered with shadows)
GlobalFX
JumpLighting
JAreaA
JAreaB
JAreaC
JareaD
etc…

If I render my JumpPersist, with my JumpLighting, i get clean lighting. Then I can go back to MyGamePersist, and load in JumpPersist and all the sub levels it needs to work properly and my lighting still is error free because i am rendering my lighting for each level area separately. This reduces loads of rendering time an as i bring on a small team to work on this game is essential.

As Mark Watney says, here is the rub.

In my MyGamePersist level, I have to then unload all my sub levels (which could be a hundred by the time I’m done here) every time i want to render. This is easy when I’m working in a sublevel like JumpPersist or BluePersist, because they are contained.

However when I delete all the sub levels that are not part of my core level to render my shadows, I then have to rebuild all my Levels in my core game, adding existing level individually.

Here’s how I think it should work (and incidentally I think this did work this way with UDK)… If I have a Level like JumpPersist loaded up, it automatically loads up all the associated sublevels that are in my JumpPersist file, so that I don’t have to keep loading tens and maybe even hundreds of levels each time i need to render.

I know bigger game companies must have a better strategy than this for working with the many levels and sub levels needed to make a game, as well as big teams.

Do I need to make a Blutility for this? Can I make a Blutility for loading up all my sub levels (and I’m not talking about doing this in game play but rather into the editor).

I know this sounds complicated, but honestly that’s the problem it shouldn’t be this complicated. The problem in my mind is that the word LEVELS implies leveling up, and the fact that a Level does not seem to bring in it’s own levels (which I’ll refer to as sub levels).

there must be a better way than tediously deleting and reloading levels over and over.

I want my game to have the MyGamePersist and it’s core levels and then every time a Persistent level is loaded it automagically loads the sub levels associated with it. I know someone will say that i can merely turn off levels, but this results in ERRORS galore because it will see anything loaded as missing updated shadows, so I’ll get the error on my screen saying hundreds of object shadows are out of date… which is untrue but still also slows the engine down.

Additionally, if there is a way to KEEP my levels (sub levels) in my Core game, then by the act of turning them off the shadow render should IGNORE them completely not give off errors that they aren’t built. Again the only way to get rid of those errors is to delete all sub levels that I don’t want to render and then bring them back after rendering.

there must be a better way.

2

It sounds like you’re trying to use baked lighting for a large world, you shouldn’t do that, as you pointed out, it’s very difficult to manage, but it will also likely end up with a huge amount of disk space used for lightmaps, that’s why games with large maps use some sort of dynamic lighting solution.

3

Darth, thanks for the feedback and joining me in this discussion.

Isn’t dynamic lighting far heavier? Dynamic lighting requires all objects to have dynamic shadows, unless someone is playing on the top of the line computer system it isn’t likely that will process well. Every time I’ve ever had a huge slow down it’s been due to dynamic lighting so this seems to go against what I understand. One of my goals with this game is to make it accessible to kids who don’t have super game systems, so I’m trying to keep the overhead low. I need shadows, but from my testing, the baked shadows are far better because they are done.

Maybe I’m missing something when it comes to dynamic lighting?

In the meantime, from my perspective seems to be more about Level loading and making this a bit more robust, not much just a little. The level loading doesn’t load sub levels for levels that someone on a team might be creating, so it takes a lot of book keeping, writing things on paper to remember what goes into a “level” so that I can load and unload in order to render shadows cleanly.

If I have a team member that has a level broken into twenty levels, I just want to load the top level into my core game, not have to check in and find out the names, it also may interfere with any persistent programming they do, that will load and unload their levels.

There has got to be a better way to approach this, I cannot imagine big game companies that use Unreal are not able to bring in a list of levels based on the top persistent level already having that list.

thanks

4

Dynamic shadows cost more real-time performance, but games have been using real-time shadows for a long time, like I said if a game is very large then it’s not really an option to use baked lighting. What you have to do though is limit the number of shadow casting lights to one at a time, usually just your directional light.

5

Thanks Darth. I will take that in. Any ideas on the level loading though? I’m probably going to try to create a blutility to do something tomorrow when i’m in studio.

6

look this discussion has gotten off track by the idea that dynamic shadows will solve my problems, over 25 years in visual effects, video games and computer animation tells me that this is in no way my solution for a problem that is about management of my Level loads. (unless I have a team of 100 and a budget of millions of dollars, but I don’t, I’m an indie developer who has a lot of experience, but is also trying to make a game that is accessible to the kids with low level computers).

I’m looking for information about loading up levels, not on running shadow bench mark tests.

In particular I’m creating a Blutility that I can use to do this in my game. I can feed it a list of my levels, and add or remove them. This is different than unloading and loading levels in game time… I’m talking about doing this in the editor.

I’m so far not seeing a hook from Blueprints to Add existing levels. Looking for information.

7

I’ve been trying in my studio to make a Blutility (which is so hard to type because auto-correct turns that to brutality every single time at least twice).

I’m not finding a blueprint equivalent to Add Existing Level. I can add Load level, but it doesn’t do anything at all.

still looking for info, but i suspect that I will need a C++ script to Load a level that I can then access from Blueprints (For my Blutility script). This would be such a clean way to work on my game levels in separate places, right now the management is such a big time drag every in every way it is one of my biggest time drains in developing my games, because often i have to unload many levels just so i can test my core game, and render the shadows correctly. Then I have to rebuild by scratch loading my levels and populating my level list. The only way i can keep it straight is by being sure to PREFIX everything, so my BLUE level needs that prefix, and IT level has that as a prefix or rather IT_.

if anyone has ideas on creating a function I can access from blueprints while in editor mode (not game mode) to Add Levels, that would be really helpful.

I’m hoping this is something we can get built into core Unreal at some point, it feels like a huge part of level management.

8

ON WORLD COMPOSITION :

I have spent the rest of the week trying to get World Composition up and running in my game. This has been somewhat confusing because the documentation on World Composition is unclear in areas.

For starters : They say you can add levels to “Layers” which had me bring up the Layers Window and wonder why Layers was not talking to Levels at all for a day.

The photos in the documentation don’t actually specify where to create these Layers with the “Layers” button (which is merely a Plus symbol in the World Composition window). So that is partly why this is so unclear.

Other Problems :

Once in World Composition it sees all Maps in your game, whether junk or not. So putting them in Layers (some of which I named Test_Layer or OFF_LAYER) is essential for sorting.

When you bring up your game as a new session, it doesn’t remember what Levels you had previously already loaded and were working on. This may not sound like a big deal, but when there are twenty to forty levels in your core game, it’s a lot of turning things on just to get started again. So it doesn’t remember.

If you were unfortunate enough to try to hit the “Play” button without sorting (i did this many times) then EVERYTHING in your maps, everything that is a MAP will get turned on as long as it’s in the default range to you. This means my game kept crashing and I had no idea what was happening. My thought was that my nodes in Blueprint that would stream things in, would still control everything, but no… if your Levels are in Uncategorized, then they ALL LOAD UP.

This had me scratching my head for quite awhile.

I tested this system out in small games that I made just for testing, and I still have no idea why it wasn’t working at all. I would remove my sub levels, and switch to World Composition and nothing would happen AT ALL. That meant I had to go back to my full game and test this, which is a major drag when trying to figure something out for the first time and get a beat on what the heck it is doing.

Once you create a LAYER you cannot edit the parameters, i.e. the distance that turns that layer on, or even if it doesn’t use Streaming Distance at all. Once created if you change your mind you have to create a new Layer, and i’m pretty sure you cannot delete layers… which in development time for a game is going to lead to a lot of junk and confusion.

BENEFITS :

I’m not sure yet fully. I’ve sorted all my Levels into Layers, I’m testing my shadow renders right now, and will continue to test for the different actual game levels and see how this goes. Again the goal is to render shadows and once combined, have ZERO shadow errors. Again this isn’t about repressing shadow errors, as any shadow errors from my experience will result in SLOWING the game down dramatically. Most people may be unaware of this, but having tested my game on not very good computer systems, I am keenly aware of where the slowdowns are.

PROGNOSIS :

hopeful. I like the idea of World Composition but this needs to be fleshed out a bit more with the users in mind.

9

OUTCOME :

I ran my tests. The way I work is that each game level has it’s own Persistent level. So Blue_Persist, and Red_Persist for instance. I load up game levels like Blue_Persist (which loads it’s own lighting and the sub levels it uses). I render shadows and save.

I render shadows in my core game, and then I didn’t even have to re-compile my game, because the World Composition seems to be working well.

My gameplay is error free (shadow-wise).

It took mere minutes to re-render the shadows associated with each level.

Feeling happy with my new setup, took a few days to wrangle it like this, but it will make a huge difference in the time I take to render, and will keep my game together even if I fiddle with individual levels as much as I like, without breaking my game to do so.

It was a week spent well in my studio that will save all kinds of time and lead to a better game, happier developer.

10

FOLLOWUP :

Ugh… i feel like someone making journal entries about their process making an indie game. I was swept away by my core game being error free without going into the other levels, blue_Persist, and Red_Persist etc…

So I triggered their load-up, and again was hit with the wall of bad shadows. Bad shadows in my experience is NOT just about errors. It is about impacting the game play time, but worse it results in objects flickering on and off. The only way i can get things to snap back on is to jump. I’m not sure why, but it’s unplayable as such.

I am testing to make sure I don’t have lights, making shadows overlapping various layers. This may result in me removing all lights at some point except my sunlight just to get past this error (or at least get it off the list).

I am following the wisdom of Tim Hobson, who talked about rendering shadows with separate streamed levels. I have done the following as prescribed, but this is still a SERIOUS problem holding me up once again moving forward.

i really wish i could talk to a team developing and using the World Composition, there must be others running into this. This can’t be so difficult that it gets broken by one rogue team member doing something wrong.

------- Snippet from the thread with Tim Hobson ------------

In order to do this, you will need to have 4 levels using your example.

Persistent Level: Do not put anything in here that you don’t want to be visible or baked in any other sub-levels

Level 1: Meshes and Light Red

Level 2: Meshes and Light Green

Level 3: Meshes and Light Blue

11

You’ll struggle to find anybody using World Composition and static lighting. I’m not entirely sure I follow what’s going on, but I will say this: Dynamic Shadows are okay and more than useable, even on a low-end device. We have two dynamic light sources on a mobile game for the entire scene, and the performance of those is the least of our worries.

If anything, in a large world your game will probably run considerably better using Dynamic Shadows because you won’t be streaming lightmaps in and out of memory all the time. It’s also much easier to manage in terms of lightmass building, which simply does not play nicely when you start using streaming levels and/or world composition. You can’t (without heavy workarounds) have different lighting styles for different streaming levels, it just doesn’t work - and you’ll eventually want an entire render farm to do lighting builds.