Announcement

Collapse
No announcement yet.

Where are you supposed to store .udatasmith files?

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    Where are you supposed to store .udatasmith files?

    Prompted by this response here, I had to ask.

    We have two different PDM systems for our CAD files, depending on whether they're in Solidworks or Catia. But files that get exported for sending to suppliers go inside a network share (or a cloud share).

    So far, we have put our .udatasmith files on that same network volume, but I recently discovered two things:
    1. If a .udatasmith file gets moved, even "re-import with new file" stops working (thankfully a bug that I got confirmed).
    2. You should not put .udatasmith files in your Unreal Projects project folder (actually this was the first thing I did when I got unreal, since this is the way Unity worked).
    So where do they go? If you share a project among several people via Perforce, and one of them wants to re-tesselate a .udatasmith file (or even a .step file), where is Unreal supposed to look for it, if not on a network share?

    In a AAA game company, for example, where are textures that get imported into Unreal stored?

    (Several people have commented that linking directly to a network share is bad, but ours is version controlled just like our PDM systems, so that's why I never understood the traditional Unreal workflow.)
    Last edited by roberteker; 10-07-2019, 10:16 AM.

    #2
    Since it was my post you are linking to, I feel obliged to try to explain

    Unreal has several layers over how it works with regards to assets, what actually gets worked on inside the editor. When you import a file, be it a Datasmith file or texture, as you are aware it makes it's own 'Unrealified' version as a .uasset file, (or multiple files, in the case of most .udatasmith imports). The originating file then becomes somewhat superfluous; it can be removed, deleted, changed etc. but it has no impact on the version in Unreal. The sole concession to this, is that assets inside Unreal get a field with the path to where they came from, so you can reimport to update them as necessary.

    But it does not stop there. In a similar vein to, say, your scratch disk in Photoshop, Unreal also has a Derived Data Cache that contains the data you are _actually_ working on at a given point in the editor; this is often not a small chunk of disk space. When you first import an asset, or open a project, Unreal has to build this Derived Data Cache (DDC), which is why it can take a while to open a project for the first time. But it is made entirely of derived data that can be regenerated whenever is needed, as so it's not the most crucial thing in the world to protect. Why do I mention this? I'll get to that in a moment...

    Now, there are a number of ways bigger groups work collaboratively on a game or project, but usually the individual texture files and similar non-code related assets, while being worked on, don't actually live in or around the Unreal project directory. They will be elsewhere, under their own system for versioning or snapshotting or the rest of it as needed. That also will often involve a traditional file server, a mapped network drive, and similar. But they don't directly interact with the Unreal project beyond being imported; it is the imported .uasset file that is checked in, not the original .psd file. That way they are saved, archived, sent off for outside work etc. without having any impact on the Unreal project.

    (It is true that there are some that do put it under version control with the Unreal project, but the important takeaway is that they do not have to. Depends a lot on how often the files are iterated over).

    Given it's rather direct (and intermediary ) usage in the Unreal workflow, I can certain see why it might get a bit upset with .udatasmith files living on the network vs locally for import, (In the same way that building the project can have the occasional issue doing so off the network vs local), but re-tessellating does not in fact look back to the .udatasmith file at all. (https://docs.unrealengine.com/en-US/...ion/index.html ) As far as Unreal goes, the only time it will ever look back to this file, to my understanding, is to reimport. As such, you can happily toss the .udatasmith file away after that if you wish; if something happens to the .uasset files, you would usually reload from the checked in copy in your version control, rather than from the datasmith file, (assuming the import was OK in the first place).

    If you need to preserve specific versions of the .udatasmith data, (for instance, as a snapshot to show exactly what the state of the files where when you sent them to a supplier), then that of course is a separate thing, but it is outside the Unreal project workflow: it can live wherever is most appropriate for the use it is being put to. But there aren't many reasons to keep them around other than that; like the Derived Data Cache, it's assumed that if something very odd happens, you can regenerate them from the source. This is to say nothing of incompatibility between Datasmith versions...

    Now, you _can_ have your project's Derived Data Cache live on a network share; that way, only one person needs to do the crunching of the assets, and that first slow load up, and everyone can benefit. This is especially handy when there are lots of imports involved; just be aware that this can also be a not-small amount of data, as well as the fact that over a network is not as fast as locally. It's also not advised to snapshot, backup, or version control this cache either.

    Charlie Morris
    Technology Research Analyst

    Rogers Stirk Harbour + Partners
    www.rsh-p.com

    Comment


      #3
      Thank you for the explanation.

      Re-tesselation was a bad example, I guess. However, we often add or move parts outside of Unreal and then re-import the .udatasmith file to update the geometry in Unreal, so we do keep our .udatasmith (or .step) files around (on network volumes).

      Comment

      Working...
      X