Landscape memory size on disk - questions

Development World Creation
,
1

Hi,

I’m developing an open-world game and yesterday I made my first attempts at importing a landscape from World Machine to UE4.

As my game design asks for huge levels, I was concerned with not only the landscapes visuals (which looked 100% ok for me in UE4, even without textures yet) but with the memory sizes on disk, loading times etc. I followed some nice links with guidelines for importing terrains from WM into UE4:

https://forums.unrealengine.com/deve…ss=&viewfull=1
https://forums.unrealengine.com/deve…d-Composition=
https://docs.unrealengine.com/latest…dscape/Custom/
https://docs.unrealengine.com/latest…landscapesizes
https://docs.unrealengine.com/latest…ion/index.html

All of the links above were very helpful. So I picked up a scene I had here in WM and changed some of its settings to fit the exporting<->importing standards. I exported a 8129 x 8129 (resolution) terrain heightmap in PNG 16-bit, as recommended - the terrain was set to 8.129 x 8.129 km in WM.

Then, in UE4 I followed the settings for landscape suggested in the fourth link above (recommended landscape sizes):

Overall size, Quads/section, Sections/component, Component size Total components: [TABLE]

8129x8129
127
4(2x2)
254x254
1024 (32x32)

It went smoothly into UE4, no errors - this was great, for a first try.
The thing is: I checked the map size after saving it and it was 433.558 Mb on disk.

- So my first question is: is that a standard size on disk for a landscape of 8 x 8 km? (8.129 x 8.129 km, to be more specific)

I then tried importing another WM terrain into UE4, a 2017 x 2017 one, with same average resolution (1m / pixel). 2.017 x 2.017 km in WM. It takes 26.55 Mb on disk. So, I made some rough calculations, approximate values, and it gave me ~2.6 Mb / km in each side (some more, but this is close).
2.6 x 8 km = 20.8
20.8 x 20.8 = 432.64 (not 433 Mb, but close, ok to have an idea of sizes)

**- My second question: **is this kind of a standard relationship in Mb on disk <–> Km of landscape in UE4 (if using this average resolution from WM of 1m / pixel)? And if so, is there any way we can reduce this size on disk without reducing the resolution? I need to ask this before going to reduce it.
1m / pixel (1m / terrain quad) felt ok in UE4. Less than that I fear may feel blocky, specially in sudden changes in relief.

Thanks in advance. Any help will be very appreciated. Some screenshots attached to help see the results. Nothing special, only messing with terrain resolution etc.

2

I just saved an almost empty level (New Level > Default), to check if this huge size (433.558 Mb) was only because of the landscape actor and indeed it is. The default level here, without any edit, just saved, gives me** 622 Kb **on disk.
I’m using version 4.16.

I’m thinking here… This size is ok for a mesh that would be 8129 x 8129 vertices (= 66.81 Million verts). It seems to me kind of optimized. But… what if this could be generated only when running the installed game on the client machine? To make downloads shorter / faster.
The 8K pixels heightmap file used to generate the mesh is taking here 63.2 Mb on disk. Not a small one, but seven times smaller than the mesh seems to be in the level file (~433 Mb).

The 2K heightmap PNG file is taking here 3.41 Mb versus 26.55 Mb of the level file (repeating: with only the landscape added).

3

Well… its a terrain in the end when it comes to dealing with them maybe you should try using some LOD on the Terrain but of course this will cost you its detail in exchange or use Occlusion.

You mean as in like by just putting that Terrain alone the level became large in size? Your going to have to come up with some brilliant methods to get around this.

Also don’t forget Unreal Engine 4 uses a load of Memory if your not careful with this it will bleed the RAM “Depending on what you have” I’m curious exactly how much is the
triangle count on the terrain?

4

That’s overkill right there I was attempting to recreate that terrain and it was already going high on RAM usage above 3GB and more also with the textures you want to add
to the terrain you typically cant go over 2400x2400 in Unreal in my game I have higher than 4000x4000 and some are higher than that and it scales it down so if you want to keep the texture detail you will need to keep that in mind Unreal has its limit based of what I’m seeing in my project.

It will only get bigger on your end I’m afraid.

5

Ernesto, thanks for taking the time.

I sure plan to use landscape pieces with lower resolution (much lower) for the distant vistas, for the ones the player won’t walk over. But for the ones he’ll actually explore, it unfortunately needs to look nice and display a standard resolution. One that won’t feel noticeable.
If you’re talking about using some kind of dynamic LOD for the landscape components, the landscape actor in UE4 already takes care of that, of swapping different LODs on demand based on the camera frustum / viewport. It even takes care of dynamically changing tessellation based on distance from camera, surface normal angle etc. Probably you know that.

Yeah, only added the terrain. The other stuff I added is only going to add to the GPU cost (volumetric fog, skylight etc). I deleted some small stuff in the level here after reading your reply, to check it better. It went from 433.558 to 432.958 Mb on disk. Only thing I left there, aside from the landscape actor, was what comes with the “default” new map: sun directional light, player start, floor (a cube static mesh) and skysphere.

The 8129 x 8129 pixels heightmap (PNG, 16-bit) has 8129 x 8129 vertices, according to the “landscape recommended sizes” on the docs.
So, if 4 verts make for a face, and a face equals 2 triangles…
But each component’s section doesn’t share vertices with neighbor sections, so the math gets a bit more complex.

From the docs:

6

Yeah, Ernesto. I’m taking note of your input here. Later I need to try to load many landscape tiles of 505 x 505 and 1009 x 1009, to see if UE4 deals differently (more optimized, smaller on disk) when using many small landscape tiles instead of one big piece of landscape, amounting to the same area in km.
I’m focusing on memory on disk here, because it was what I was investigating. But you’re right, sure RAM usage will also be of concern.

Yeah, you’re probably right regarding those limits. I just tested the “worst case” scenario (heavier one piece of landscape) to test UE4 limits in this regard. And also because the docs list it (8129 x 8129) among the recommended sizes, as the heavier one acceptable.
Some of my levels are designed to be even bigger than 8 km x 8 km. I’ll need to use a good amount of tiles…
Actually, the level designs (top-down layouts) are not squares. They have irregular shapes, “arms” extending in irregular directions, like this (but more complex):

http://imgal.ru/usrfile/img/figuryi-tetris.jpg

So it’s not actually something like 8 km x 8 km. I’ll need to calculate the total area with all the missing pieces, extending ones etc.

7

You will have to be careful with the way you build your levels, to chose code over blueprints,

I work with BIGDATA and Optimization so performance is EVERYTHING to me However if you do not work with Optimization in a technical level like I do than you have to be careful with how you build your game other cheats you can use is Dynamic lighting so you wont have to bake anything on the level because that alone is a nightmare.

You know try to be smart with how you build the level and work with the terrain don’t be like the others who thing they can do everything simply because they depend too much on Big Memory.

8

I planned since the beginning to use dynamic lighting. Also because the game will have time-of-day changing.

I see your point. You’re right. I’m not making an open-world game because of unrealistic, exhagerated ambitions. It’s because the game story and gameplay asks for big landscapes - the player most of the time travels the landscapes fast, using vehicles. And it’s a game about traveling. So the size of the levels is directly linked to the essence of the game. And I don’t plan to fill every meter with lots of encounters and experiences. Nothing like Skyrim, where you meet a wolf, a sabertooth tiger every 20 to 50 meters - don’t get me wrong, Skyrim is a great game, I played it a lot.
But my design is more oriented towards a different experience, less intense in encounters.

I was just thinking here that I may need later to hire/recruit a programmer to make code, maybe a plugin, that would have some means of making the landscapes use less disk space, at least on the downloaded file, Then it can (re)generate the landscape actors during install on the client machine, if this be possible.

9

Landscape is less expensive than static mesh in most cases, but it still calculates the tris/vertexes. So you still need to treat it with caution. If your constantly filling your screen with monstrous mountains, tesselated paint, and foliage youll notice extreme drops in fps, and image glitches like early 3d games. My advice to you for extremely detailed mountains that wont function in game is background prop cutouts. You can easily do this by changing your sky box to a solid color, take high res screenshot in editor, and using a color erasing program of your choice. I would also find a tutorial on auto height based landscape material thats instanced. There are plenty of tutorials on how to make one that doesnt cost much, but changes, and tiles textures by angle, and height. You’ll have a bit less control, but a much faster result on looking at your performance in general. It’s well worth having those tools in your box anyways. This level is only 15 mb with barely any height changes, and only solid colors. That includes the Third person content

10

Thanks for your tips, thadkinsjr.

FPS is doing fine here. As I said above, I didn’t use terrain tiling, world composition etc yet. I was just testing the most extreme/worst case scenario, using a single 8129 x 8129 landscape actor. My main question here for this thread was regarding average size on disk for each size of landscape actor. So that I can make more accurate calculations, optimize things with the download size of the game in mind etc.

It’s a good advice. But this is something I’m not worried about right now. Sure I’ll use much simpler LODs or cutouts for distant background mountains. But right now I’m testing for the actual playable, navigable areas.
I appreciate you taking the time, really, but this example in your image above doesn’t apply for my case. It’s too low res for the style of my game. Maybe for distant backdrop mountains. But thanks, anyway!

11

I would really like to hear some tips, info and advices from the Epic devs on this subject. :slight_smile:

12

that’s a nice mountain. Would look good for a sprite cutout. I think its more better to just drag the detailed mountain out into the engine and texture it, then after its spent an hour building all the lighting and so take a snapshot pic of it and save it and load it in the gimp, cut out the mountain, and then import it as a paper sprite so you can just use the cutout of the mountain to to build your distant 3d mountains ranges with. You don’t need to have real 3d mountains in the level that take up millions and millions of polys unless you’re insane. but I do think its a good idea to drag a mountain terrain in the engine and texture it all so you can prepare some nice paper sprites for your project to use.

I think with games like Mass Effect II, they used paper cutouts for the distant alien backgrounds so when the player moved closer and further away in the level, the mountains would shrink or grow in the level to maintain the illusion. I think that’s why some of the map levels in Mass Effect were restricted to allow you to only roam in and around close near the buildings, while the areas close near the mountains in the distance were all inaccessible to the player to stop the playing from getting too close to them.

These cutouts however allowed the illusion that you was down on this big vast planet that stretched out for miles but the reality is that you only had a small tiny play area to move around in and this design kept the engine down to a good FPS and maintained the illusion, but when you saw that your play area was restricted and you could not get up near those big mountains and the game kept you mostly close around the buildings, the game soon felt fake because they put guard rails and other barriers up around the area to stop the player wandering out to break that feeling up that this is not real but might be just a bunch of cutouts, they also threw in a couple of levels with the vehicles using
a real terrain.

13

Yeah, I’m also planning to use barriers to restrict the player walkable areas, disguised as natural barriers (cliffs etc).
And, yes, I also plan to use cutouts for the distant vistas in the horizon etc.
Nice thoughts on this, tozan, regarding Mass Effect etc. It’s the kind of very useful tricks to have, to give this illusion of huge vistas, as you pointed out.

But I guess we’re getting a little sidetracked here. This post is more about tips on loading landscape actors, resolutions vs memory on disk, how to optimize this (if possible) without losing too much resolution in the terrain (specially the quads right next and under the player, distant mountains are not much of a problem)…
I mean, those mountains in the pics are areas that the player will actually traverse, explore. I attached some more images to help show this. The player will actually be able to climb to some peaks. Of course at some point their way will be blocked, to limit the level boundaries etc.

(Ah, the character in the pics, as you can see, is just the third person example character, with some tweaks just for fun and for the sake of practicing blueprints etc. I’ve put some burning flames under his feet with example content, after changing his default jump settings to make he fly a little, to reach some peaks more easily, look somewhaaat like iron man, hehehe. No relation to my game).

14

So can that big huge mountain terrain of yours be level streamed?, That’s what I want to know… Ok, In the old days of Red Faction and Doom portal face brushes were used to tell the engine which areas of the level map to draw and what not to draw and with that system you had to
place all your portals face brushes FACING AWAY from each other to avoid causing the Hall of Mirrors Effect…

If you did not place in enough portals within in your level, then the engine WOULD CRASH because it would try to render
too many things at once so if you was doing a large map, you had to put quite a few face portals in it. And also and if you
didn’t kept also your object clutter count all down to an acceptable level within each portal area and had too many objects
that would crash the engine too or cause the game go all funny in that portal and start having objects losing their collision values
and so on. So you got to be careful with the portals.

So Unreal Engine might be similar I don’t know. So i think things have to be balanced with this engine if doing a open world level
with it. I only see two options either a 45 degee camera top-down 3d view so you don’t see any drawing, and you can have a
flatter landscape or deserts, Or if you use the normal third person eye level camera which allows you to see out for miles, then
your landscape needs to be raised up in places so you can’t see the edges of the rendering being done in the far distance
or use fog to hide it.

15

I use this for testing the actual size on my disk. I just ran a lake, and valley up for demonstration. So it literally only takes a minute for me to import a terrain map, and click auto landscape material. Insert my texturesamples, and Tile to approximate detail, and then save level. Now I can find out a rough approximation of how much data my terrain is going to use as you said (Size on disk.) Without foliage, props, and code. I wasn’t concerned with the fps of your actual terrain non textured. I use this to check my data size. It gives me an almost instant test. I dont have to sit around painting to find out I have 300mb of built data without props.

16

Yeah, thadkinsjr. I was testing for the same thing. Memory on disk, in relation to resolution, Specially resolution right next to the player - because resolution in the far distance seems to hold well enough even with lesser resolution. I just tried now here using a resolution of 2017 x 2017, and it’s holding nicely in terms of details, realism. I’ve attached a screenshot using this res. I’m still going to try 1009 x 1009, same world size, to see if it holds well in terms of details.
I will stress, because maybe I wasn’t clear enough above: I was first using 8129 x 8129 to begin with to test for the heavier case on disk, memory etc. Now I’m gradually going down in resolution.

Also: yes, of course I’ll use tiling for the terrain pieces. I still need to test this, it is in my plans. From advices I’ve read, probably 1009 x 1009 or 505 x 505 tiles.

I still would like to hear from the Epic devs any info and tips on this.

17

want to know this too

18

Really the initial question is relative. (size on disk)
It depends on poly count and complexity of the landscape. Tessellation on or off? Every option has a + or - to it and will effectively add to the complexity of the landscape.
Really what your asking is “is my car fast?”. Sure, for what it is, why not?
What can you do to optimize? Doesn’t matter, you seem to think an untextured landscape with no code and no actors on it is how it will some how end-up. This landscape, in its 8k glory and size, will end your project. I can 100% tell you that right now. No need to say i am crazy, i will wait for you to have lots of code, millions of actors, foliage, etc etc to come back and say “why is my performance failing?”.
Your initial “why is this so big on disk”, like i said, is relative to the amount of poly/verts on your landscape. Not that its a single actor/mesh. Take that thought process out of your mind. When a 3D designer makes something and says “its only got 500 million poly, i dont know why this is so big in file size” >> i can tell you why.

  • sticking to topic *

Less resolution will lower the poly/vert count. Making less size on disk and over-all performance will get better. This doesn’t need an Epic developer to tell you the tips/tricks you need to do. I am really not sure what you are looking for to be answered here but,

  1. Lower the res of your map
  2. Don’t use Tess
  3. Use auto material in a simple way. The more math it needs to do (if and or but) the more performance you will lose.
  4. Depending on landscape size, and, your using a big area i can see, use LODs, lots of them.
  5. If your not allowing a player to use the outer-bounds of the map, cut out those mountains and use some mesh for them with some simple material. One thing you want to stay away from is detailing the areas the player wont see, or, cant get to. No need to have millions of grass mesh if the player isn’t ever going to see it. That’s just millions of wasted overhead.

If i missed “the point” of the thread, please, let me know what the actual question is and i will be more then happy to assist you.
Open world tiles? If you have the player ability to see beyond its current tile, it will see the end of the tile. It doesn’t always work out as intended. Tiled maps, in theory, sound great. Test it out, stand on a mountain. Notice no tiles are loaded in but the one you are standing on. Talk about breaking “reality” in your game, right? Even if you extend it out, it still fails. Unless you postprocess your players ability to even see that far. Or load them all in, then, what was the point of the tiles?

19

[quote=“AP_Studios, post:#Post:0x000055984c968fb8, topic:99546”]

Really the initial question is relative. (size on disk)
/QUOTE]

Hi AP_Studios, thank you for your significant answer!)

I have not big UE landscape but his weight is 436882.562 KB, material with 3 simple texture.
How I can reduce this, but keep opportunity make some holes and no AAA caves)

then, could you give some advice - how to better start landscape for open infinity world, world composition, or there is limitations?

20

Thanks for the reply, AP_Studios. But… did you read my last post above? I was not asking “is my car fast?”. It would be a very subjective question, and also not specific enough for one who is looking for technical answers with numbers, tables etc.

As I said above

I should also mention that I’ve read more then once the doc page that specify how much video memory (and memory on disk) each texture resolution takes.

Again, I’m not that un-informed about the relationship between file size and mesh complexity. I’m a 3d artist generalist with some 15 years of experience with maya, zbrush, realflow etc etc, also used to code in MEL, python, actionscript, beginner level in C++ etc etc. Of course, more vertex position vectors, normals etc to store, more memory the file will take.

Also:

You made a wrong assumption about what I was thinking. I precisely tested it without textures, landscape layers etc yet to see how much memory only the heightmap and mesh would take, to **make calculations **before even thinking about adding more stuff. But how could I know before asking more experienced users (and specially the guys who implemented the tech) if there was or not any kind of optimization in the engine code to make the landscapes use less memory?

What I was asking for was just a yes or no to 2 questions. I’ll copy paste then from the beginning of the post here:

Also, again:

I don’t intend to waste memory loading tiles for distant vistas the player won’t be walking over at all. Why should I? But again, as I said, my game concept asks for an open world. It’s intrinsic part of its gameplay mechanics. This is the reason I’m doing all these performance tests before creating the actual levels. Yes, sure I’ll use tricks for the vistas the player won’t be able to traverse.
Thanks for taking the time and all the advices, tips, really.