Datasmith Feature Requests

Is it possible to request a more streamlined terrain to landscape conversion? Currently exporting terrain as a datasmith is great but lacking the functionality of a landscape object. I am trying to incorporate UE into my workflow for large scale visualizations using real world survey data and it would be great if I could import graded civil 3d terrain objects into and then export them as datasmith and have them functions as a landscape in UE4. If there is already a plan for this or I have missed another suggested workflow to accomplish this my apologies.
Thanks

This is on our list of “to do”, just that there is a lot of stuff ahead of it. We’re aware of the problem.

I second the greggotsch request but from CityEngine end. CityEngine utilizes raster DEMs but landscape offers lot more functionality.

Hello all,
I work at a construction firm and have been following several companies who are getting into the VR realm. They are uploading their models into software to be able to walk the job sites and give their foreman a better understanding of what they are going to be building. My company currently uses AutoCAD MEP and Fabrication MEP, both produce dwg files and I noticed that Unreal studios doesn’t support those files yet. Is there any way to get those file types available for the engine?

Your path is probably through 3Ds . We could consider dwg, but your version would require object enablers that only autodesk provides that we don’t have access to.

I can be wrong it’s been a long time I was exposed to those technical aspects…

It would be great if you could report back if you can get this happening from AutoCAD->3ds ->Unreal

I don’t know if someone asked for this before me. In my pipeline I use an hybrid system: datasmith for architectural elements and non repetitive elements modeled in Revit and csv data reading for positioning my blueprint actors as their counterparts are in Revit for library ones (laboratory analyzers, benches, fixtures, luminaries and so on).

For Datasmith importing it would be fine to have some kind of voxelization option for big walls, ceilings and floors. I think so to avoid 4K lighmaps for these elements that as well in many circumstances 4K doesn’t give enough quality. For example, if elements bigger than a 10 meters cube are encountered, then they would be divided in so 10 meters voxels as needed, and convenient lightmapped. This is not only good for lightmapping calculations but also for distributing meshes for calculations on swarm agents in a balanced manner.

What do you think?. I know a studio that is doing something similar in house plugin.

Sounds like a good idea, maybe some kind of integrated/streamlined mesh splitting within a single mesh asset to allow multiple lightmap UVs ?

Yes, this is the idea: to avoid too big items that need big lightmaps. It could be achieved modeling in Revit (or whatever BIM app) for balanced dimensions with this in mind, for example creating floors and ceilings in a per room basis, or walls partitioned every x meters, but this is not the way for BIM modeling. An authomatic parameterized voxelization could solve this.

Yes, we’ve recently talked about the need for this. We have a bunch of “worst case” Lightmass baking scenes we received from customers that we want to throw at the new Lightmas. If you want to provide some problematic examples, feel free.

Testing datasmith in 4.20 & SketchUp 2018.

  • is it possible to add an ‘export selected’ option in SketchUp? I have lots of helper objects in the scene -hidden- that still all get exported to US. Lots of manual cleaning to do in US which takes quite some time.
  • could the geometry be flattened into 1 SM object if they are in the same group? At the moment, if you have a group with several ‘clusters’ of geometry, each cluster of connected faces is converted into a separate SM in US. Makes the scene hierarchy rather messy.
  • could it be smoothing is not supported at the moment?

I dont know but it seems like all guides and tutorials and everything require us to use 3dsmax while I do not use it nor do I want to install it just to to some basic stuff. It would be great if the UVW-nromals or whatever it does could be integrated just fine without the use of 3ds .

We are continuing to work on the SketchUp workflow. You should have solutions to these coming. Not sure about smoothing, but unless that is something special, it is supported. Perhaps you should log a bug.
Report Unreal Studio bugs here: https://unrealstudiohelp.epicgames.com/s/article/How-do-I-report-a-bug

The export visible will be supported next build. Then scenes will be your workflow to decide what to export or not.

for smoothing I can check. Can you send us a skp model that shows the issue?

Hi @pf_breton , thanks for the reply.
Great the export visible will be an option in the near future. Smoothing works after all - my mistake.

I have a few suggestions if I may:

  1. could the object names in the geometries folder be the same as the object names in SketchUp and in the World Outliner? At the moment they don’t match.

  2. I added a small picture showing the issue of meshes in groups in SketchUp that get split up into separate meshes in US (imho not the expected behavior - I would already have made separate groups in SKetchUp myself if I would like them to get split up). A fix would clean up (shorten) the Oultiner graph in US a lot.


Exactly. I think that the whole DataSmith usefulness would be optimal to everyone that will pay the monthly fee if it was able to just convert each and every supported native 3D software data format without any requirement to rent or buy the license to any of them. When 3D models are bought on other online shops and the fbx is missing, incomplete or corrupted and there is a native file for another 3D software then DataSmith would be useful indeed if it was able to just convert those to UE4 with no need to do anything else. And obviously with no conversion issues or mistakes. Unless Epic Games would add in the fee a special license to some high end expensive 3D software editor but I don’t think that that would ever happen. So DataSmith being able to convert everything easily would be the best option.

I want a new feature: [right click on actor] -> [Reset to origional position]
(Just like 3ds filelink a dwg file)

The next version of the plugin will most likely improve on that. There are however details in how Sketchup stores data that we also respect that can change things a bit.

See this image 2018-08-01_0954

In the above model, I have 2 components and 2 groups. In Unreal, the actor names will respect what has been set in the Sketchup outliner. Note that a user can customize names in Sketchup that don’t get stored are the Group/Component definition level so we are taking account of that.

For components, they will point to the same meshes instances so we have one asset for many actors: the asset will use the component definition name, while the actor will use the outliner name.

That is by design. When several separate connected meshes sit inside the same level in the hierarchy, we preserve their separation. Grouping all faces as one mesh will create other issues, like gigantic lightmap uvs, improper culling etc. When working with unreal, we have to unlearn old habits from other renderers (like combining meshes by material or by layer) : this works against you in a realtime engine like Unreal.

The real issue is “too many actors from sketchup” in general.

In the future we will provide tools to regroup objects in a more smarter way to reduce drawcalls while respecting instancing. We have a prototype of that done in python that achieves this, I have attached it if you want to pay with it (no promises made to its usefulness). The downside of that method is that it will break re-import so we are thinking of a way to do it as a pre-import step.


import re
import unreal

# content folder where to put merged geometry
content_folder = "/Game/"

# utility function to get the full hierarchy of an actor
def get_actor_hierarchy(actor):
    result = ]
    list = actor.get_attached_actors()
    while list:
        a = list.pop()
        result.append(a)
        for child in a.get_attached_actors():
            list.append(child)
    return result

# retrieves all Actor in levels having the SU.GUID.XXXX tag
level_actors = unreal.EditorLevelLibrary.get_all_level_actors()
guid_actors = {}
for actor in level_actors:
    tags = [str(tag) for tag in actor.tags if str(tag).startswith('SU.GUID.')]
    if len(tags) > 0:
        guid = tags[0][8:]
        if not guid in guid_actors:
            guid_actors[guid] = ]
        guid_actors[guid].append(actor)

# handle nested components by using only the highest level actors for a guid
for guid, actors in guid_actors.iteritems():
    actors_to_remove = ]
    for actor in actors:
        parent = actor.get_attach_parent_actor()
        has_ancestor_in_roots = False
        while not parent is None:
            if len([tag for tag in parent.tags if str(tag).startswith('SU.GUID.')]) > 0:
                has_ancestor_in_roots = True
                break
            parent = parent.get_attach_parent_actor()
        if has_ancestor_in_roots:
            actors_to_remove.append(actor)
    for actor in actors_to_remove:
        actors.remove(actor)

# merge and instanciate
for guid, actors in guid_actors.iteritems():
    print "GUID: " + guid
    if len(actors) == 0:
        continue

    actor = actors[0]
    # retrieves the list of static meshes to merge
    actors_to_merge = [a for a in [actor] + get_actor_hierarchy(actor) if a.__class__ == unreal.StaticMeshActor]

    # special case where 0 or 1 actor to merge
    if len(actors_to_merge) < 2:
        for old_actor in actors:
            if len(actors_to_merge) == 1:
                # keep static mesh actors and reparent them
                static_mesh_actor = [a for a in [old_actor] + get_actor_hierarchy(old_actor) if a.__class__ == unreal.StaticMeshActor][0]
                static_mesh_actor.attach_to_actor(old_actor.get_attach_parent_actor(), unreal.Name(), unreal.AttachmentRule.KEEP_WORLD, unreal.AttachmentRule.KEEP_WORLD, unreal.AttachmentRule.KEEP_WORLD, False)
            # delete unnecessary nodes
            actors_to_delete = [a for a in [old_actor] + get_actor_hierarchy(old_actor) if a.__class__ != unreal.StaticMeshActor]
            for delete_actor in actors_to_delete:
                print "deleting actor " + delete_actor.get_actor_label()
                unreal.EditorLevelLibrary.destroy_actor(delete_actor)
        continue

    print "merging all static meshes under " + actor.get_actor_label()
    backup_transform = actor.get_actor_transform()
    actor.set_actor_transform(unreal.Transform(), False, False)

    cleaned_actor_name = re.sub('^-a-zA-Z0-9_]+', '_', actor.get_actor_label())
    new_path_name = content_folder + cleaned_actor_name
    print "new path name " + new_path_name

    merge_options = unreal.EditorScriptingMergeStaticMeshActorsOptions()
    merge_options.base_package_name = new_path_name
    merge_options.destroy_source_actors = False
    merge_options.mesh_merging_settings.bake_vertex_data_to_mesh = False
    merge_options.mesh_merging_settings.computed_light_map_resolution = False
    merge_options.mesh_merging_settings.generate_light_map_uv = False
    merge_options.mesh_merging_settings.lod_selection_type = unreal.MeshLODSelectionType.ALL_LO_DS
    #   merge_options.mesh_merging_settings.lod_selection_type = unreal.MeshLODSelectionType.SPECIFIC_LOD
    #   merge_options.mesh_merging_settings.specific_lod = 0
    # only if single LOD level is merged
    #   merge_options.mesh_merging_settings.merge_materials = True
    merge_options.mesh_merging_settings.merge_physics_data = True
    merge_options.mesh_merging_settings.pivot_point_at_zero = True
    merge_options.mesh_merging_settings.specific_lod = 0
    merge_options.mesh_merging_settings.use_landscape_culling = False
    merge_options.mesh_merging_settings.use_texture_binning = False
    merge_options.mesh_merging_settings.use_vertex_data_for_baking_material = False
    merge_options.new_actor_label = cleaned_actor_name
    merge_options.spawn_merged_actor = True

    # merge and retrieve first instance
    merged_actor = unreal.EditorLevelLibrary.merge_static_mesh_actors(actors_to_merge, merge_options)
    merged_actor.set_actor_transform(backup_transform, False, False)
    merged_mesh = merged_actor.static_mesh_component.static_mesh

    # instanciate all other components instances
    for old_actor in actors:
        if old_actor == actor:
            new_actor = merged_actor
        else:
            new_actor = unreal.EditorLevelLibrary.spawn_actor_from_class(unreal.StaticMeshActor, unreal.Vector(0, 0, 0), unreal.Rotator(0, 0, 0))
            new_actor.set_actor_transform(old_actor.get_actor_transform(), False, False)
            new_actor.static_mesh_component.set_static_mesh(merged_mesh)
        new_actor.set_actor_label(old_actor.get_actor_label())
        new_actor.attach_to_actor(old_actor.get_attach_parent_actor(), unreal.Name(), unreal.AttachmentRule.KEEP_WORLD, unreal.AttachmentRule.KEEP_WORLD, unreal.AttachmentRule.KEEP_WORLD, False)

        print "  deleting hierarchy of " + old_actor.get_actor_label()
        actors_to_delete = [old_actor] + get_actor_hierarchy(old_actor)
        for delete_actor in actors_to_delete:
            print "   deleting actor " + delete_actor.get_actor_label()
            unreal.EditorLevelLibrary.destroy_actor(delete_actor)


In 4.20 you right click on an actor and see a datasmith menu, then reset overrides. That will reset the actor to the original state from the datasmith scene.
We don’t separate position from other properties however.

My guess would be; if the modeller in SketchUp takes the effort to name an object (even if the component already has a name) - he/she must have a reason for that and it would be nice if the name would persist into US.

In principal I agree or the next thing people will complain about is the slow performance of US.
But maybe there could be a threshold before an object gets split up into multiple parts? No need to split up objects if the total face count of the group doesn’t exceed 200 faces or so(or give people a way to control this setting)? For people who create clean and low poly models (the best way to keep SketchUp fast and responsive) this would be nice.

Interesting code. I will have a look at it once I figure out how to run Python in UE 4.20. Last time I looked I couln’t find any official documentation.

Breaking re-import could be bad once you start to add interaction to your objects in the scene (a rotating door for instance) ?

Kind regards,