How to manage load times of large Datasmith levels and why aren't my models instancing? - 3DsMAX

[anonymous_user_712d4dfc]

Hello,

I have two issues that I’d love some guidance on - the first is the length of time it takes to import and open levels with Datasmith files in them.
The second is Datasmith not creating instances of duplicated meshes in max.

I am currently working on a project that requires my team to load 2 commercial buildings + furniture at around 200million tris each into ue4 for VR.
I know that’s a lot of tris, but I assumed that datasmith would deal with that level of density given it’s designed to work with pre-rendered assets.
Exporting to Datasmith is quite fast, however importing and opening the levels in UE4 take close to an hour if not longer on some computers. It speeds up if it isn’t the first time opening the level, but if someone has made an amend to the level - it seems like it reprocesses each time it opens.

I understand the import process can be long, and that isn’t a deal breaker for me. It is just the opening of levels that is very difficult to work with.

I have tested breaking them into sub levels just to see the difference even though this isn’t viable for my project and have found that if I export 1 out of 5 floors in DS it still takes close to half an hour to open the level.

What are my options for managing these huge files? Is anyone else having the same issues and how do you deal with it?

I also have issues with Datasmith not recognising that the same asset has been duplicated multiple times in Max. It still brings in every asset as individual meshes, not instanced meshes which is bloating the project.
(Note: These max files are not developed in house, they have been received from a third party so I can’t confirm what approach they have taken. We are also a maya studio so we are a bit out of our depth working with these files.)

Does ue4 recognise if an asset is the same by its name in the scene explorer? If so, that would make sense as to why it’s not instancing as the naming in these files is questionable. So then would the solution require me to simply rename all the objects?

Any tips or tricks for managing huge scenes would be really appreciated. I am almost considering reverting to the old method of individually exporting the meshes and reassembling in UE4 but I’m hopeful that datasmith can be useful for us.

Thank you in advance and let me know if I can provide any screenshots or other information.

My machine specs are as follows:
2x 1080 ti
128gb ram
2x Xeon CPU E5-2697
M2 EVO 960 HDD

The standard office machine is around:
1080 - 1080 ti
32gb ram
i7 5960x - 7700k
Varied HDD’s

[pf_breton (Pierre-Felix Breton)]

@anonymous_user_712d4dfc

I also have issues with Datasmith not recognising that the same asset has been duplicated multiple times in Max. It still brings in every asset as individual meshes, not instanced meshes which is bloating the project.

If they are copies in 3ds max (not instances) they they will be copies in UE. You can confirm this in 3ds max with the select instances command. We are thinking of re-instancing them using some form of a hash calculation but that is not a piece of work done yet.

[pf_breton (Pierre-Felix Breton)]

@anonymous_user_712d4dfc

Does ue4 recognise if an asset is the same by its name in the scene explorer? If so, that would make sense as to why it’s not instancing as the naming in these files is questionable. So then would the solution require me to simply rename all the objects?

We don’t rely on assets names to determine if something is an instance or not - that breaks too easily. We instead keep track of unique ids provided by 3ds Max and rely on 3ds max instancing system

[anonymous_user_712d4dfc]

@pf_breton

We don’t rely on assets names to determine if something is an instance or not - that breaks too easily. We instead keep track of unique ids provided by 3ds Max and rely on 3ds max instancing system

Hi Pierre-Felix,
Thank you for your repsonse.
This is the exact answer to my second question. Having looked at the files and using the select instances command it does appear that not everything is instanced. Thanks again for clearing that up.

I’ll sit tight and see if anyone can help with the first question on the large files issue.

[david.gillespie]

Hi,
Has anyone got to the bottom of the large file issue at all? I’ve been working with a number of large datasmith files and they’ve ground opening times of levels down to a halt. It’s literally taken over an hour to open a level with a datasmith file inside. It’s a city model, exported (in this case from City Engine but via OBJ via direct Rhino import) and is all mesh geometry.

• It takes a couple of mins to open the original Rhino file however the level.
• It took approx 40 mins to import the Rhino file into Unreal via Datasmith
• It took at least an hour to open the a level with this Datasmith actor in

[UE4_Archviz]

@anonymous_user_712d4dfc

Hello,

I have two issues that I’d love some guidance on - the first is the length of time it takes to import and open levels with Datasmith files in them.
The second is Datasmith not creating instances of duplicated meshes in max.

I am currently working on a project that requires my team to load 2 commercial buildings + furniture at around 200million tris each into ue4 for VR.
I know that’s a lot of tris, but I assumed that datasmith would deal with that level of density given it’s designed to work with pre-rendered assets.
Exporting to Datasmith is quite fast, however importing and opening the levels in UE4 take close to an hour if not longer on some computers. It speeds up if it isn’t the first time opening the level, but if someone has made an amend to the level - it seems like it reprocesses each time it opens.

I understand the import process can be long, and that isn’t a deal breaker for me. It is just the opening of levels that is very difficult to work with.

I have tested breaking them into sub levels just to see the difference even though this isn’t viable for my project and have found that if I export 1 out of 5 floors in DS it still takes close to half an hour to open the level.

What are my options for managing these huge files? Is anyone else having the same issues and how do you deal with it?

I also have issues with Datasmith not recognising that the same asset has been duplicated multiple times in Max. It still brings in every asset as individual meshes, not instanced meshes which is bloating the project.
(Note: These max files are not developed in house, they have been received from a third party so I can’t confirm what approach they have taken. We are also a maya studio so we are a bit out of our depth working with these files.)

Does ue4 recognise if an asset is the same by its name in the scene explorer? If so, that would make sense as to why it’s not instancing as the naming in these files is questionable. So then would the solution require me to simply rename all the objects?

Any tips or tricks for managing huge scenes would be really appreciated. I am almost considering reverting to the old method of individually exporting the meshes and reassembling in UE4 but I’m hopeful that datasmith can be useful for us.

Thank you in advance and let me know if I can provide any screenshots or other information.

My machine specs are as follows:
2x 1080 ti
128gb ram
2x Xeon CPU E5-2697
M2 EVO 960 HDD

The standard office machine is around:
1080 - 1080 ti
32gb ram
i7 5960x - 7700k
Varied HDD’s

please cross check is meshes are instance in 3dsmax. if yes then export instance meshes in one batch. datasmith handle them very well, but if u export instance meshes in parts then instance connection will be break. and they will import in UE as individual asset.even material connection also break. so its wise to export instances altogether.