Ingesting entire complex 3D city model meshes into UE

Development World Creation
, , ,
1

Hi,

I need some sort of advice/direction regarding a technical challenge I have:
How to use large number of complex meshes into UE wile keep a descent FPS?
How to display large number of complex meshes in VR mode?

Just a bit of background: I work for a 3D mapping company and we generate complex 3D city models from aerial photogrammetry.
We do this kind of things: http://aero3dpro.com.au/adelaide_CBD (check the interactive WebGL viewer).

Just to give you an idea the all 3D city model is made of 650 individual mesh tiles with a total size of 32 Gb.
An individual OBJ or FBX tile size is between 70Mb and 115Mb and includes 3 to 4 texture files (8192x80192). We are talking about 1M faces per tiles. All buildings, trees, cars, street furniture are part of a continuous mesh.

In the geospatial industry, many software have been developed to deal with this kind of heavy data by using adaptive quadtree or quadtree LoD systems. so we know it is possible to get a very smooth interactive experience.( like the WebGL viewer in the link above).
Most of the videos in this channel have been captured from real-time viewer using adaptive quadtree LoD:https://www.youtube.com/c/aerometrexau
Many viewers can also display the data in stereo with adaptive quadtree LoD 3D model such as this Cesium viewer: https://sample.aero3dpro.com.au/NightCliffe_2016/App/index.html

What I would like to achieve is to get those 3D models to work smoothly in Unreal Engine and specifically in VR mode.
This is were I’m stuck.

Unreal Engine can load FBX or OBJ with up to 4 simple LoD.
I’ve managed to load a large dataset and can fly around interactively in game mode but the FPS is not good enough.
&t=3s (this video has been captured with matinee).

When I ingest the simple FBX LoD tiles I need to setup the right texture manually for each LoD. it is a long process but at the end, it works.

However visualizing even one unique mesh tile in VR mode is not working properly and the movements create a very jittery view due to the complexity of the mesh (around 1M faces at the highest LoD). it works only with simpler mesh tiles.

I have a demo dataset with 6 tiles in FBX format with simple LoD, quadtree LoD and adaptive tree LoD.
https://s3-ap-southeast-2.amazonaws.com/perma-bucket-aerometrex/aero3dpro_Adelaide_CBD_sample/aero3dpro_FBX_10simple_LoD.zip
https://s3-ap-southeast-2.amazonaws.com/perma-bucket-aerometrex/aero3dpro_Adelaide_CBD_sample/aero3dpro_FBX_4LoD_embedded.zip
https://s3-ap-southeast-2.amazonaws.com/perma-bucket-aerometrex/aero3dpro_Adelaide_CBD_sample/aero3dpro_FBX_adaptive_quadtree_LoD.zip
https://s3-ap-southeast-2.amazonaws.com/perma-bucket-aerometrex/aero3dpro_Adelaide_CBD_sample/aero3dpro_FBX_quadtree_LoD.zip

There is a definite bridge to be build between the geospatial industry and the gaming industry.
interactive geospatial viewers can deal with complex data but lack game engine development tools and VR capability (and are not open-source for the majority) while game engine such as UE can’t deal with complex mesh data but have amazing gameplay, effects and VR capabilities.

I’m not an programmer so I apologize if the questions seem inaccurate or if I’m missing the point:

Could this challenge be won if it was possible to load FBX with an adaptive quatree LoD into UE?
What would be the process to implement an FBX loader with up to 10 quadtree or adaptive quadtree LoD insteal of 4simple LoD?
How much would it cost to rewrite an FBX loader? Are there any other UE components which would need to be modified?

Any suggestions/ideas/comment are more than welcome.

2

For that you really need an adaptive quadtree system, otherwise it will take too much work to set up a system using the current LOD system. The game engine just isn’t designed for that purpose because has to manage many assets at a small scale. Not sure how difficult it might be to add an adaptive quadtree system to the engine though.

3

Cheers.
Which elements of UE needs to be modified? the FBX loader is one. what else. I’m not familliar with the programming aspect of UE but I’m just trying to get some direction on that.

4

I’ve been playing with hierarchical LoD and things are improving a lot.

5

Hello,
my team is interested in achieving VR movement in aerial photogrammetry 3D mesh,
and the things mentioned about LODs’ might be in my interest.
Would you mind bringing some light to the technical achievements you’ve made
and any good advice on making movement without things ive considered:(using complex collision on a huge 3d city, or putting some simplified box collider on the bottom of the mesh or using navmesh which results in a higher ground than needed),all aint seem quite right.
Any help would be appreciated thanks :slight_smile:

6

If you’ve got some very high resolution photogrammetry of something like a city then you’ll need to optimize things to make it run well, and everything depends on what you’re going to do with it. If you’re just going to show some movement to view the mesh then you probably don’t need collision (unless you want to avoid allowing the user to move through the mesh). With something big and complex, you can’t do automatic collision generation, so you’d need to create a collision mesh yourself.

Ideally as far as LOD’s go, you’d want to split up the mesh into pieces and then have LOD’s for each of those.

7

I’m interested in viewing the mesh like travelling inside the city so i think using collision is inevitable.
i’ve read a bit about " bounding volume hierarchy", and making some kind of custom colliders using this technique and use the name conventions as written in the ue4 documentation for colliders, But i didn’t really dig deep into that.
would you say that is the right direction to go into? other algorithms to making colliders by script?

8

It depends on what you mean by “traveling inside the city”

There’s not going to be any fast way to have colliders automatically made for a photogrammetry mesh, it has to analyze the mesh and with so many vertices it’s going to have trouble. The best thing you can do is make the collision meshes yourself.

9

i mean like actually walking in the streets, like the “City Guided Tour” with VR videos in youtube
simple example of city guided tour : https://www.youtube.com/watch?v=m9EClKA1VeQ
just without the AI element of people walking near you, just an empty street.
** But with the ability to walk

I’m not interested in an easy way, just an automated script it doesn’t have to be fast, but i need it to be a script
so i can change the terrains(city) used easily by using the script and not having an actual person who will make it by hand.
if you could just point me in the right direction of an automated algorithm or paper that would be lovely.
thanks again for all your comments so far, very helpful :slight_smile:

10

Are there no plugins, algorithms for such things out of ue4 scope?