UE5 - World Partition - Massive Map Sizes?

Hawk_UK · March 27, 2022, 3:06pm

Just a question about the sizing of terrain in the new UE5 World Partition:

Do we still have to be restricted by the UE4 World Composition rules about the sizing of terrain/landscapes?? Particularly the “Recommended Landscape Sizes” from the UE4 landscape technical guide?

I ask this because as far as I understand it in UE5’s “World Partition”, we can now just import a landscape of any size and World Partition will automatically handle all the tileing, HLODS, and streaming in of those tiles, etc, etc, according to the settings we put into the World Partition options.

Basically I’m asking if in UE5’s “World Partition” we still need to take note and use the “Recommended Landscape Sizes” from the UE4 Landscape Technical Guide? Or can we now create any size landscape model from real world heightmaps/data and just import them into UE5 World Partition? Obviously taking into account the limits as far as SSD space required for that data.

It just appears to me that all the tutorials on UE5 World Partition are still using the old UE4 World Composition dev pipeline of using the UE4 Landscape size restrictions from the UE4 Landscape Technical guide instead of taking advantage, as I understand it, of the new capabilities of World Partition Landscape Size possibilities?

Hawk_UK · March 28, 2022, 4:47pm

C’mon guys… Has no one any clue on this question about World Partition?

peterbengtson · March 28, 2022, 10:38pm

I’d really like to know the answer to this as well.

Hawk_UK · March 29, 2022, 3:40pm

Yes indeed mate… This is really frustrating that no one here in this forum is able to give any genuine indication or resolution on this question… Pretty disappointing to be honest mate… could it be that no developer really knows?? I honestly doubt that… so C’mon guys, someone here in this forum must know the answer to this question, surely?

briligg · June 10, 2022, 8:43pm

I need to work with a map that is 41 km on a side, and I can’t trim it much as it’s based on a real place. This is something I’ve found that I hope is a solution:

An imported heightmap larger than 4 km across needs to be broken into tiles, with a specific naming convention. Each tile must have a suffix that indicates its position in the map by showing what number it is in the x direction and what number it is in the y direction, as in the image

DemenzunMedia · June 15, 2022, 2:04am

UE5 World Partition is still limited to the same Recommended Landscape Sizes from the documentation, and the equivalent large worlds based on those dimensions, such as 16x16 tiles of 1009x1009 tile size, or 8x8 tiles of 2017x2017 tile size.

Note that the calculations for the tiled worlds is different than World Composition and I’m still working on calculating what it is, since Epic has not documented it, so that I can add that to my TerreSculptor software. But for now the standard dimensions when tiled seem to work.

This is because the Landscape actor uses power-of-two textures under the hood to manage all of the Sections and LODs. So these same design restrictions are imposed in UE5 as well.
Read the Landscape Technical Guide page to see how Sections and Components relate to GPU textures, and follow those guidelines when creating any terrains.

The UE5 Landscape interface for creating New Landscapes is limited to a maximum of 8161x8161, the same as UE4, when manually creating a new terrain.
To create terrains larger than 8km x 8km requires that you import an XY Tile Set.
So computer system main memory and video memory are determining factors for how large of a terrain world you can create without running out of memory.
Importing the tiles and creating the cells uses main memory and GPU memory, and for World Partition the amount of memory used is higher than UE4 World Composition.

The basic rule of what you will require for a computer for main memory and video memory for importing and loading all cells for various world sizes is:
8km x 8km = 32GB memory and RTX-3050 8GB
16km x 16km = 64GB memory and RTX-3080 10GB
32km x 32km = 128GB memory and RTX-3090 24GB
64km x 64km = 512GB memory and RTX-8000 48GB

I run an R9-5950X, 128GB, RTX-3090 and the largest world tile set that I can import before running out of memory and failing with an error, is 44km x 44km or 2000 sq km.

DemenzunMedia · June 15, 2022, 5:39am

I have spent the past few days digging into the World Partition system, and I have figured out their new World Partition world Landscape sizes. They are different than World Composition, my guess is to make the Section size a larger texture for fewer drawcalls on large terrains.

I will be adding the information that I have determined for valid world sizes to the Unreal Engine Landscape dialog in the TerreSculptor Terrain Software, and also put this information into the upcoming video series on my YouTube channel regarding World Partition.

So I would recommend using the correct recommended World Partition sizes for large worlds.

Rossoe · June 15, 2022, 11:23pm

This is really interesting - thanks for the figures, I’ve been struggling to convert tiled terrain to Word partition with my GPU memory running out pretty fast.

I’ve got 64GB PC ram, and 8GB on a 3070 GPU
But interestingly when trying to import terrains and convert in UE5 my shared GPU memory took a hit and anything in the region of 16km terrain simply failed:

However when I tested importing an even larger 24km terrain (3 x 3 - 8129 tiles) in UE4, save it and then open that level in UE5 to trigger the conversion steps, then finally convert to World partition the memory consumption seems less aggressive with hardly any shared memory being used:

so I’m not sure if the WP conversion step is just more optimised for UE4 conversions?

Performance wise I’m getting 8.31 FPS with all tiles loaded in runtime:

And 14.12 FPS with half tiles loaded:

DemenzunMedia · June 16, 2022, 9:36pm

“the memory consumption seems less aggressive”

My first guess would be that importing tiles directly into UE5 World Partition requires a lot of GPU memory to create the Landscape Section textures, and that will all be done on the GPU.
Whereas simply converting an existing UE4 Level will already have those Section textures created, they just have to be moved over into LandscapeStreamingProxy actors in UE5 (the textures don’t have to be re-created).
So the amount of video memory would be really low on a conversion versus an import since it isn’t creating new terrain Section MIP LOD Textures. If you follow what I mean.

Any Windows software will have available to it both Dedicated and Shared video memory to use for the texture creation step in importing.
The Dedicated memory is how much memory is physically on the video card, such as 8GB, 10GB, 12GB, 24GB, etc.
The Shared memory is calculated as 50% of the computer system’s installed main memory. So if you have a system with 64GB of main memory, the Shared video memory will be 50% of that or 32GB.
This means that on a system with 64GB of main memory and a 10GB RTX-3080, the total amount of video memory available for rendering textures etc is 32GB + 10GB = 42GB.
Depending on the amount of video memory required for the import process, you might run out of video memory during LandscapeStreamingProxy texture creation, in which case the only way to resolve that is to either purchase a video card with more Dedicated memory, or increase the system’s main memory in order to increase the Shared memory amount.

Note however, that UE5 World Partition has totally different world sizes than UE4 World Composition, so converting any Landscape from UE4 to UE5 will either crop or pad the Landscape world size. Which might be a bad thing depending on what you want and what exactly happens in the conversion. I personally wouldn’t do it.

UE4 World Composition used tiles of Recommended Landscape Sizes as defined in the UE4 Landscape documentation, such as 8x8 tiles of 1009x1009 tile sizes with shared edges. which would result in terrain Section sizes of either 63 or 127, and Components that are either 1x1 or 2x2 Sections.
World Partition does NOT do this! It uses an entirely new sizing method, based on Section sizes of 511 (not 63 or 127). My guess is to get the number of Drawcalls lower for rendering large terrains in World Partition.
This means that totally different Landscape sizes will have to be used for World Partition.

You can calculate the new valid World Partition sizes as (511 x N) - (N-1).
So for example, some of the valid World Partition resolutions are:

16321
24481
28561
32641
44881
48961

This also means that to create a tile set for import, you will have to determine what tile multiplier dimensions are valid. Most World Partition resolutions only have one tile size set value available for them, because they are such odd numbers.

For example:

16321 is only divisible by 19 × 859
24481 is a prime number
28561 is divisible by 13 × 2197 or 169 × 169
32641 is only divisible by 7 × 4663
44881 is only divisible by 37 × 1213
48961 is only divisible by 11 × 4451

Rossoe · June 17, 2022, 12:10am

Thanks for detailing those figures - it’s super useful.
and I also just read your other thread - UE5.0.2 Landscape Mode Broken? - #4 by DemenzunMedia

My previous project used 5x5 tiles of 4033 res 20165 - is that good to convert as is? or should I expect some cropping or padding on those figures?

And I was just starting a new project with 3x3 tiles of 8129 res of 24387, should I be trying to use 24481 now? but I’ve fallen for the one that’s a prime number, am I now stuffed due to limited divisibility? I’m trying to avoid cropping/padding if at all possible.

What you mentioned about the memory makes sense. I’m technically not using world composition in UE4, but I am importing terrain tiles in using a plugin called TerraForm Pro, Do you think there a possible workaround of still going via UE4 and then converting into UE5 to avoid the RAM limitations? as I don’t think I’m going to be able to upgrade my memory any time soon, but I would really benefit from working on a 24km terrain for this project.

Out of interest do you use the command line option for WP converter or GUI option?

Do you think the FPS I mentioned in my earlier post is to be expected, or is there some scope for improvement using appropriate new tile multiplier dimensions, or perhaps other config tweaks?

DemenzunMedia · June 17, 2022, 12:38am

For World Composition, 4033 and 8129 tile sizes should not be used for any production games, they cause hitching on medium level computers such as an i7-6800K/GTX-1060 even without any foliage or meshes.
I detail all of this in the lengthy eight-part World Composition Video Series on my YouTube channel.

A heightmap of 20165 will be padded out to 20401.
So there will be wasted space around the edges.

A heightmap of 24387 will become padded in UE5 World Partition to 24481 resolution.
You can get away with this but you will essentially have junk on the edges of the Landscape that are wasted. AFAIK this isn’t visibly rendered, but will be wasting the texture memory and texture sizes.

You don’t have to worry about the 24481 divisibility, UE5 can import large PNG files up to 32767x32767 without having to use tiles.
This is one of the new features that pretty much no one knows about.
So far I think I’m the only one who knows except for maybe some Epic employees.
So you can import a 24481x24481 PNG heightmap directly without splitting it into tiles.

What happens in TerraForm Pro if you try to create a UE4 Landscape that is 24481x24481?
Will it create it correctly?
If that works, then just use a valid UE5 World Partition resolution in your UE4 project, then convert that, it should convert without any cropping or padding.

If you need to know all of the valid World Partition large resolutions, you can calculate them as (511 x N) - (N - 1).
Or you can download the free TerreSculptor software that I develop and simply look on the Unreal Landscape Sizes dialog to get a full list.

I have always used the command line conversion option for WC to WP.
However, I have found that it is just way faster to create the Landscape in UE5.
I converted a 32km x 32km 1024 sq km 32x32 1009x1009 WC to WP using the command line, and on my R9-5950X, ASUS ROG X570, 128GB, RTX-3080 system it took over 26 hours to convert. Not fun.
The conversion also added padding to the Landscape size, of course.
I recently upgraded to an RTX-3090 because I was tired of running out of video memory.

The FPS performance that you are getting sounds about right.
Attempting to load all cells on a 24km Landscape requires a lot of power.
Typically you are only going to have between 3km and 5km view distance in-game, so the performance should be way better when playing the Level.
I would set the Loading Range to around 300,000 (3km) to 500,000 (5km) and run a performance test in the PIE viewport and see what your performance is there.

MostHost_LA · June 17, 2022, 2:33am

You all should know that the world composition system has been said by official channels to be “dead” for over 4 years.
Since ue4.22 or right around that time.

Why is it still in the engine?
Because epic acknowledges how BAD the system is compared to its competitors, but hasn’t actually bothered doing anything about it - As per usual.
(It’s arguably better to be able to do things on a bad system than to not be able to do things because the syatem was removed).

Re performance:

If you completely re-write (from source) to disable the retarded component merge system they put in place (which was originally based on diatance until .18. worked a lot better then but still has the same baseline issue. Its not how you do reductions.) - then, all modern machines, tosters included can perform ok with 8K tiles.

4k tiles are the standard on most professional projects. They do require heavy optimization to work on 1060s with around 30fps.

Remeber, if you want performance:

Do not use unreal.
even if you disregard 1, Nothing will ever beat converting the landacape to a static mesh, and using proper instancing to populate the grass/vegetation (Procedural components can help too on this).

Re recommended sizes:

Those are just math based.
Meaning its the optimal size for the math to work efficiently when the landscape(s) are displayed.

It does not extend to what works for your overall draw call count.

Each landscape has a max of # of components drawcall count to render.
Each mesh has essentially 1 drawcall to render.

You can already see why converting landscape to a mesh usually gives better performance.

On sizes above 22km^2 it may be best to use 8k tiles.
Less tiles = less impostors. Less impostors = less additional draw calls.
Less extra = more vegetation/better assets.

8k also has larger components.
You can try to use the 8K variant, strip it down to 4km^2 and effectively have half the components on the equivalent 4k tile with nearly the same effective size.
Since this results in half the drawcalls, it can drastically improve performance (and uses nearly the exact same height maps).

Any landscape below 12km^2 doesn’t necessarily even need world composition anymore.
4k size tiles cut to half components (total size 2k) will perform well in this case.

Additional notes:
Don’t think for a second that world composition and world origin shifting are all you need to set up multiplayer.
Theres plenty of core code issues with the level loading system, in any version of the engine.
(And even competitors dont do a lot better on this aspect)

DemenzunMedia · June 17, 2022, 3:08am

We are discussing the World Partition system here.

The one poster is simply discussing World Composition because he can create large landscapes there without running out of memory, and then convert those to World Partition. Doing that has the issues that I discussed in my post.
The creation and conversion has nothing to do with World Composition render performance or drawcalls or Section render size that you are going on about.

Isn’t “efficient” the entire point of it?
Why use Landscape sizes that are not efficient?

Game developers use Landscape because of the toolset and features and addons.
Most game developers are not going to be concerned with Landscape versus terrain mesh render comparisons. Converting to a mesh is beyond what most game developers want or need.

MostHost_LA · June 17, 2022, 3:44am

No, if you wanted efficient you just would never use unreal. Period.

The fact you are using it and reading here means that you are as of yet unaware, or you just decided not to care about performance.

That, or you are a victim of someone else’s bad decision - and you decided to make the best of it, attempting to undesrand how stuff works to squize some milege out of an otherwise dead horse.

In which case half the drawcalls is a better choiche in 99.9% of cases.

UE5 even built from source with a very specifically gutted rendering pipeline is just sheer underperforming trash when compared to others.
It appears that Epic’s assumption of performance is that you should render 800x600 and upsample to 2k.
That said, an honest question.
Do you find any improvement in the new system?
I do not. It has the same fatal flaws.

mr-newell · June 17, 2022, 6:54pm

@DemenzunMedia I just downloaded TerreSculptor to give it a try; what’s the proper way to import tiles and re-export them as an Unreal World Partition-compatible landscape?

What I’m currently stuck on is:

Import Tiles…
Modify > Resample (what’s the correct size to pick? My landscape is 3750Wx5000H)
No matter what I choose it creates big seams between each tile; is there a way to avoid this?

DemenzunMedia · June 17, 2022, 8:06pm

If you are importing tiles to stitch into a single heightmap, you would use the Import Tiles on the file menu.
That feature has limited DEM file support and doesn’t support shared edges yet.
If you are getting a proper stitched heightmap, then that step is working.
When selecting tiles for stitching, they typically go X+ left to right and Y+ bottom to top.

Before you use the New or Resample tools, you should go into the Settings, Dimension tab, and enable the Unreal Engine dimensions, then you will get the Unreal Engine Sizes dialog available on those tools.
You should also go into the Settings, Units and set the Units to the proper Engine that you are using.

Once you have done that, then on the Resample dialog, click on the Unreal Engine button, choose the proper resolution from the available Unreal Engine set, then resample the heightmap.
To use Recommended Landscape sizes which would work in your case since you are working with a small terrain, that would be 3025 x 4033.
If you prefer to go with the next bigger sizes instead of shrinking the terrain, go with 4033 x 6097.
Choose that on the Recommended Sizes.

If you are getting big seams in UE5 then it will probably be because you are not using a valid UE5 World Partition resolution.

mr-newell · June 18, 2022, 3:06am

Yeah I followed those instructions; the seams are appearing in TerraSculpt itself after the resample and not during the Unreal import. I will perhaps try to find another data source; this came from the USGS though so it should be fine.

DemenzunMedia · June 18, 2022, 4:33am

It sounds like the tiles probably have shared edges on them in that case.
The Import Tile function doesn’t handle shared edges yet.
The Import Tile tool is still under development.

Also, make sure that you are using the Resample modifier and not the Downsample modifier, those are two totally different things. Downsample can cause seams on resizes that are close to the same size.

You may wish to simply look for some good single 1 meter GeoTIF data, those files are 10,012 x 10,012 in size, so no tiling is required for a single terrain of that size for what you are looking for.

See the video on my channel:
“Real World Landscapes”
https://youtu.be/aY_NlVd2Dpk

MostHost_LA · June 18, 2022, 4:41am

Pass it through a GIS pgram and ensure that the final output is png16.

Data is data, so if you modify or alter the images in any way without knowing what you are doing (which again, is Data manipulation, not a graphics thing) you are prone to get issues/bad seams/terracing, on pretty much on anything you use.

If you don’t know how to use a Gis program, try Grass GIS.
I recomend it over others simply because it can stich several tiles togeter and rasterize to peoduce a unified height file in a timely fashion. its also opensource, free, and it works via python, so you can script things like image chopping.

There are other similarly free editors out there you could try.
And UI editors that can allow you to stretch with radial blur and export tiles like Nip2…

You could also just re-acquire the data off usgs and try using it without any modifications to it.
The Tiff file is 16Bit already.

If it fails, you could try Terrain.party since it still seems to be up (it was limited to 20m DEM). It used to expot as PNG16 directly.

Rossoe · June 22, 2022, 10:33pm

We are now shooting for 16k tile after testing larger sizes, performance is really not workable.

It’s a shame as I was hoping to make use of World Partition for much larger terrains. I’ll have to await further optimisation from Epic before pushing for bigger resolutions.

Examples of FPS at various resolutions (Heightmap was at 1mtr resolution):