Foliage + Lightmaps is broken/bugged/dysfunctional - Detailed explanation

General Feedback & Requests
1

So,

I’ve been looking into this issue for a the past few days now and have come to the conclusion that either foliage with baked static lighting is currently fundamentally broken due to what seems like some design oversights, or that I’m missing something with how it works altogether.

My scenario is this:

  • I have a large landscape with approximately 31k trees, currently consisting of 1 tree model.

  • The tree model has lightmap Uvs carefully laid out for a minimum of a 32 x 32 px lightmap, which the model is set to by default in the static mesh editor.

  • Tested and baked individually with the default lightmap size of 32px, the tree lights correctly as expected.

Upon painting and baking lighting for the tree on the landscape, I am receiving the following warning:

**

**

The result of this is that the lighting for the trees looks very bad, seemingly due to the lightmaps now being reduced below 32px. Looking at the warning, it tells me to “consider reducing the component’s lightmap resolution or number of mesh instances in this component”

Now, to me this is unexpected - my landscape is made up of 64 components, each with approximately 484 trees, my Packed Light and Shadow Map Texture Size is set to 1024 in World Settings, so it seems like there should be plenty of room for 484 32x32 lightmap atlases. (if my calculations are correct, you can fit 1024 32x32 lightmaps into a 1024x1024 texture).

Regardless, I gave it the benefit of the doubt and increased Packed Light and Shadow Map Texture Size to 2048, and then 4096. The result was always the same warning + bad lighting on my trees.

I did some further investigation, looking at the baked lightmaps themselves with Packed Light and Shadow Map Texture Size maxed out at 4096, and concluded that it currently tries to bake all foliage into the same atlas, regardless of any landscape component. With this and the fact that the editor references all foliage as InstancedFoliageActor_0, It seems that all foliage is treated by the lightmapper as one giant model which would be highly inefficient, especially when it comes to texture/lightmap streaming.

Here are the following things I have tried to break up the foliage into smaller components/lightmap atlases:

  • Duplicating the model and painting the copies as different foliage types.

  • Creating multiple Foliage Type assets with the same model and painting them as above.

  • Separating the landscape into 4 different sub-levels and re-baking (highly inefficient workflow)

None of the above worked for me, I still receive the same warnings and the same issues for all scenarios.

In Epics own documentation on Foliage Instance Meshes I found this under Lighting:

**

**
Has the cluster size been changed? Can it be changed manually? I cannot reduce the lightmap size of my trees any lower and it seems crazy that the engine would not atlas them per-component. I’ve seen dozens of threads posted about this issue with zero solutions and I think it really needs to be addressed.

Can someone offer insight or advice on this?

I’d be happy to upload an example project if necessary.

Thanks

Answerhub Post is here

2

You would probably be waaaaaay better off using things like DFAO and cascaded shadow maps or a combination of the two; than trying to lightmap 31k trees.

https://docs.unrealengine.com/latest/INT/Engine/Rendering/LightingAndShadows/DistanceFieldAmbientOcclusion/

https://docs.unrealengine.com/latest/INT/Platforms/Mobile/Lighting/HowTo/CascadedShadow/

3

The problem with DFAO and any kind of distance field/dynamic solution is that I would have to sacrifice both GI lighting and perf as well as excluding a chunk of people who don’t have the hardware required to render distance fields. Other problems revolve around the fact that many of Unreals lighting features rely heavily on Temporal AA which limits me further.

Baking lightmaps in this massive scene right now only takes about 5 minutes on my machine which is a fair trade-off for the image quality and perf I get back.

Using dynamic lighting also does not address the issue the issue, which is that foliage with lightmaps does not seem to be working correctly/as expected.

4

How is storing 31k variations of tree lightmaps more efficient? Even if they are low res, that’s still an insane amount…

In my opinion, I’d just fake a cavity/AO map into one of the texture channels, disable them from lightmapping, vertex paint the terrain near areas that need more shade and call it a day.

5

It’s actually not bad at all - in my test map the entire scene, with 31k trees plus landscape lightmaps accounts for only around 150mb, then bear in mind that if streamed properly, only a small fraction of that is loaded into memory at one time. This seems perfectly reasonable to me (when/if it works as expected).

Unfortunately AO does not replace good GI, and although manually painting in bounced light/shadows is an interesting concept, it drastically increases the time needed to light and iterate on scenes.

6

some more info regarding my test scene and model:

examples.jpg
examples.jpg1176×3860 718 KB

7

With a lightmap resolution of 32x32, you aren’t really dealing with that much quality anyways. For a box, or something relatively sharp angled, they might work just fine. For a tree, that’s going to be even the size of the player? It’s going to blow those bad lightmap artifacts right out in the open. Again, you’d be better off with either adding in a greyscale channel for faked AO/cavity or you could vertex paint the tree model’s shadows in. Either way, you could easily do away with their lightmaps entirely. Then from there, you could still project their shadows onto the world lightmap. Win:win.

And no, it doesn’t drastically increase anything if you’re vertex painting in on the landscape; with layers. Just darken the ground in that area. You could manually go around and do that in 5x5km map in the matter of hours. Just focus on areas where players will be. Either way, I’d suggest what I said in the previous paragraph.

EDIT: Looking at the pics you posted, yeah, I’d honestly do away with lightmaps on them. The resolution is definitely so low that it looks like vertex painting anyways.

8

Firstly, two points here:

  • AO does not, in any way equate to GI, and by that I mean bounced/indirect lighting, soft shadowing, accurate shadow penumbras, area lights, translucent shadows, emissive surfaces, etc. - AO is just one element of that. AO + dynamic light can in no way reproduce the same quality of lighting that you get from lightmaps/pre-calculated GI lighting. Not currently anyway.

  • 32x32 works just fine for the tree - there is enough lighting information there to accurately reproduce the GI needed (which is very important to look i’m going for) and any inaccuracies in direct lighting and shadows are bridged via a stationary direct light which allows for the best of both worlds - baked lighting and dynamic shadows.

Going fully dynamic with lighting is fine and totally legitimate if you are operating in a fully dynamic world, for my circumstance it is unnecessary; my world is mostly static as is my lighting and I want to be able to take advantage of that. Less ms spent on dynamic lighting means more room for complexity in other areas.

Painting a simple shadow under a tree is one thing, but then you have to think about all of the other aforementioned qualities that you would have to compensate for and paint in manually also - this is where authoring things turns into a time sink, especially if your scenery is continuously changing throughout development.

I hope you don’t think i’m trying to shoot your suggestions down, they are genuinely interesting and im definitely going to be considering that route if I cannot resolve this issue, but I want to explain why it’s important for me to go down the route of having baked lighting. I really wish there was a decent, cheap realtime GI solution out of the box with Unreal but as of right now that just isn’t the case. :frowning:

9

I never said AO=GI… I said that you could fake some cavity shadows, as in places like the base of the tree or branches near the trunk, with AO/cavity-like mapping. Many 3d programs have options for these. Zbrush has it built right into the render options. Even with other programs, like 3dsmax/maya, you could just throw in a skybox with a light that is at a similar angle to your scene and bake out some “AO/GI/cavity” style maps. I was simply giving you a way to fake in some shadows. You are complaining about the engine not caving to your insane needs for the amount of light mapped models you have. That’s the engine’s way of saying “Hey… You should consider some alternative options for your game’s lighting needs…”

Your trees don’t look like they have many polygons in them. Vertex painting one tree and instancing that would definitely be cheaper. You might even be able to find a way to convert the “AO/GI/cavity” style maps into vertex color information. It would need some interpolation, but it would work. Programs like Zbrush make this a breeze though where you can take a texture and “drop” it down onto the mesh, in the form of polypaint, and vice versa. That way you would have one generalized tree that could be distributed across the scene. If you needed a bit of variety for them, you could make a few variations of shadowing. Regardless, it would probably be astronomically easier than dealing with 31k lightmapped trees…

And again, you can still bake the tree’s shadows into the terrain’s lightmapping.

10

Biggest reason to have trees static is to have them cast shadow to the ground(without hidden geometry hacks). 31k * 32x32 * 6bytes per texel is just 186MB memory. That should be totally doable and texture atlas should be split to multiple entrys.

One hack would be to use dublicate trees. So each tree has just max number of instances that still fit to single 4k page.

11

You can have the trees cast shadows on the ground; without them having lightmaps…

12

I actually already used baked in AO and cavity maps, but it is insufficient.

Again, i just want to iterate that 31k lightmapped trees is not insane nor is it pushing the engine in any way; as Kalle-H points out - 31k trees with 32x32px lightmaps only amounts to 186mb - that’s the equivalent of approximately 7 2048 textures. To put that into perspective, it’s equal to two current-gen character models (with diffuse, normals, pbr textures, etc).

I shipped an Unreal 3 title for the last gen consoles that had well over 100k lightmapped objects in one level with no problems - it’s all down to how well you pack your lightmaps and stream your levels.

The fact here is that Unreal 4’s foliage lightmap atlasing is fundamentally broken - if I were to bake 31k individual tree static meshes instead I would have no issues (other than in increase in draw calls).

I had a similar idea to this but it seemed to still try to pack everything into one atlas - I’m going to try again though, perhaps splitting it into 4-5 copies of the same model will work? I will try it out and report back.:slight_smile:

13

Unfortunately it seems that having 5 copies of the mesh painted as different foliage types did not work. I made sure the equivalent amount of trees were painted (around 31k) but the only thing that has changed is that I now get 5 sets of warnings instead of one:

foliagesplitx5.JPG
foliagesplitx5.JPG1350×184 63.5 KB

Note that all the warnings refer to the same InstancedFoliageActor - this further leads me to believe that all foliage is being lumped together into one actor for some strange reason.

14

Have you tried to look source code of this?

15

Unfortunately I’m not really adept with c++ so I’m not sure I’d know where to start with that

16

Then you didn’t do it properly or you didn’t set up the materials properly. you need to make sure to adjust the w/b levels to “amplify” contrast and such.

Read up on draw calls. I’m almost positive it’s having to make a draw call for each of the lightmaps; despite the trees all being duplicate instances of each other. The engine isn’t moving whole “boxes of paper” at once, it’s having to move “pieces of paper” one by one. So the character model analogy doesn’t work.

The main take-away here is that you’re persisting on using an inefficient method, complaining that the engine needs to be fixed or something and will not take suggestions; out of confirmation/belief biases. Whatever floats your boat though…

17

It is still an issue regardless if he has 31k trees or 1k trees. In bith cases Lightmass will attempt to put ALL folaige in one texture rather than dividing them into components.

18

I ran into a similar problem with railroad ties.
Made a blueprint that places ISM instances along a spline. Above a certain number of ties, the engine would give me the same warning.
I solved it by making the bp a mix of ISM and child actor components.
Now I have a BP that spawns actors that construct ISM instances.
So inastead of 1 actor with 100 ISM instances, I have 10 child actors with 10 ISM instances.
Works fine :slight_smile:

Another idea:
Put half of the trees in a new sub level.
As lighting is built per level, lightmass will definitely not putting that in the same lightmap.

19

I’m not sure what you mean by the pieces of paper analogy, but I will do some tests on draw calls vs instances + lightmaps to confirm if this is correct. I’m also going to be exploring more dynamic solutions this week.

That’s a cool way of approaching it, will definitely keep this in mind as I’m planning of doing something similar.

I experimented with this a while back but it didn’t seem to work - I think though I need to go back and try again in a more controlled test environment - I’ll let you know what turns up!

I’m also thinking of hiring a coder to look into this a little more since it’s kind of a big deal to my project.

20

Ok so I learned some interesting things today:

**Firstly draw calls **- I did tests with a scene that has around 1500 trees, comparing a fully baked scene and a fully dynamic one:

drawcalls.jpg
drawcalls.jpg1876×1776 566 KB

My conclusion is close to what I expected; foliage with lightmaps is still instanced and therefore has low draw calls. The dynamic scene has more draw calls due to the dynamic shadow pass (although not hugely different). So It’s safe to say using lightmaps will not increase draw calls.

Secondly, component sizes and lightmaps - I learned that foliage is not packed in the lightmap atlas according to the landscape component it’s painted on, but via the Instance Static Mesh component. So how it works is basically this:

It will try to fit ALL instanced meshes of a certain foliage type in a single level into one atlas, regardless of their location or proximity to one another in the level. This is obviously not a particularly efficient nor streamlined way for the engine to be handling this but I have discovered that by having multiple Foliage Types (Create new asset>Miscellaneous>Foliage Type) with the same mesh component, I can effectively push one foliage mesh counts beyond the limits of a single Packed Lightmap Atlas.

Also it seems that moving objects or landscape components which have foliage painted on them to different sub levels helps!

So I think for now, the solution is to carefully keep track of how many Instance Static Mesh Components are in the level vs the Packed Lightmap size and break them up carefully as desribed above.

Oh and one other interesting thing I discovered is that Half of your Packed Lightmap Atlas size is used for directional lightmaps - so you will need to half the amount of foliage instances you can fit into one atlas!

Hope this is useful info!