DataTable vs UDataAsset for storing large amount of static data

[kostenickj]

Hi all,

My game is very data driven, i need to store a large amount of data with the same structure, and i’m stuck between using DataTables with structs, or a UDataAsset object. I’m curious what some more experienced developers might have to say. This is 100% static data that will never change at runtime, it will likely have 10k or more rows.

Is there any pros and cons to using each?

Does one type handle schema changes better than the other without data loss?

I wont need to store pointers to any UObjects, just primitive types and structs!

Any input on your experiences with either is appreciated!

[NickEast (Nick van Oosten)]

Technically, a Data Asset is typically a single piece of data (or a single struct), so it’s the equivalent of a single row in a Data Table (which uses a single struct as a schema, and can then be filled with all the data. Each row is basically one instance of the struct). If you have a lot of data with the same structure, basically a table/database, I recommend a Data Table.

On top of that, with a little consideration, a DataTable can store references to assets (asset strings) and I believe objects as well (basically any variable type).

As far as pros and cons go, you’re basically comparing apples and oranges AFAIK so there isn’t really any reason to, I think.

[Starbladek]

Old thread, but it’s the first one that shows up when googling “unreal data asset vs data table” so I wanted to share my experience with them.

tldr: DataTables are easier to work with, but DataAssets are much more stable (in my experience)

I’ve personally found that DataAssets are much more stable. During the multiple times I’ve tried to use Structs + DataTables, Unreal semi-frequently crashed when I tried to add/remove/change the properties in the struct, and in some rare cases, my project would become completely bricked and crash on startup everytime, to which I had to use source control to revert back to a previous version of the project. I verified that the commit that caused it to become bricked was the one where I changed a single default value in the struct used in a DataTable.

This is conjecture, but based on my own experiences and some loose research online, I believe unreal has a lot of trouble handling structs that contain user-defined properties, like non-engine blueprints and other user-defined structs. If the struct ONLY contains basic properties (like ints, floats, strings, etc), its been a lot more stable in my experience.

DataAssets, on the other hand, have been rock solid for me. I’ve never noticed any correlation between me messing with PrimaryDataAssets/DataAssets and unreal crashing, whereas with Structs/DataTables I noticed it all the time.

While they are more stable, it is slightly more cumbersome to edit the values of a particular DataAsset you want to, since you have to open the data asset directly. There is no nice table to look at and quickly change the values you want to change.

It’s also harder to get a reference to all of the data assets at runtime; DataTables have the really convenient “Get Data Table Row” and “Get Data Table Row Names” nodes, while DataAssets have… nothing, as far as I know. The only way I was able to get a reference to all of the item data assets in my project was to do some weird crap like this:

It gets the job done, but it feels really awkward to have to do. It’d be awesome if PrimaryDataAssets had a node like “Get All Data Assets” or something.

Hope this helps someone out there make up their mind on which to use

[ErikFailing (Dreanom)]

The above answer is absolutely correct. I’ve extensively used both DataAssets and DataTables in both C++ and Blueprints. Using DataTables based on blueprint defined structs is a risky endeavor that can cause your project to have to pretty rough corruption issues.

I prefer DataAssets because of how powerful they can be in combination with these two UCLASS specifiers:

All UCLASS Specifiers

Using DataAssets, you can dynamically change behavior and the available data fields with different subclasses of a UObject:

[Magiton]

I see that the Data Asset is better where it comes to stability and everything. But the user experience is more then unconfortable for me, especially when handeling bigger amount of data.

Imagine that you are doing a wizzard fighting arena game and you want each elemental spell to have a different reaction based on what material you hit.

let’s take Fireball, when it hits, I want it to spawn explosion Particle, Sound and scorched Decal. We have our default reaction.

But if you hit wood, I want to use the decal with emmisive, if I hit water, I might not need a decal at all, but I want a steam particle instead, you shoot ice, now I want water puddle decal and steam particle. Maybe I want bigger VFX when I hit oil, so we have scale for VFX and maybe some decals need to dissaper faster, so I have a timer setting there. We have 15 elemental spells and 30 unique surfaces.

Now when we have this table done for every element, we want spells be more powerfull with if casted with a staf. I guess we can go with just scaling the VFX and Decal but we want more powerful SFX for these. So, we will duplicate existing Data Assets and need to override just the SFX. The new SFX has the same name as the old one, but with “_epic” suffix.

With Data Tables, I’ll export it to Excel, add suffix, and import it back. with Data Assets I have a task for two days min.

I know there is a way to edit multiple Data Assets at once but then everything is “Multiple Values” and you don’t know what you are editing half the time

is there any good way to edit the Data Asset data outside the UE? or are the Data Tables that bad?

Thanks to anyone who read this and even more thanks if you can help!

[TheShinyHaxorus]

Just want to point out to anybody that reads this thread that you can totally do this hypothetical with Data Assets without a TON of pain, just a little. What I’m thinking is: Say you have that Fireball spell you mentioned, and you want a different decal and particle per physical material. You could make a struct containing a Decal Actor Class Reference and a Niagara System, and then in your data asset, add a variable of type Physical Surface with a map to that Struct. Then you can just fill them out per data asset (spell, in this instance) to get all the different cases, and pull the info as needed in whatever actor you are using to spawn all those effects in.

[baobao4435]

The data table BUG is fixable.

1:turn off auto save in project settings can decrease the chance to appear BUG.

2:if unfortunately your project is totally broken after changing struct used by the data table.try to adjust a random member in that struct(add a new variable or change a variable name).then save the struct.close the engine.it will prompt you asking you to save all data tables .here is important.choose DON’T save.and restart the engine.
Sounds silly.but it works for me.

[Auran131 (Auran13)]

sometimes these tricks work, sometimes they dont,
i’ve basically given up on structs/datatables outside of small simple convenient ones.

Regarding DataAssets you can bulkedit via property matrix which is even better than datatables because you can hide/show whats relevant, pin data etc

[Altrue]

I’m sorry but I have to say, it’s crazy to entertain the idea that “DataTables” vs “DataAssets” is a valid question. It’s like comparing Car vs Carpet.

They are not for the same purposes. Are you really gonna replace each row of a Data Table with a brand new Data Asset instance? It will only stay reasonable for a low number of rows. Low enough that you didn’t need a DataTable anyway. I know you can bulkedit editor objects, but that’s not a valid alternative to an actual spreadsheet software.

I’ve been using DataTables for many years now without ever encountering corruption issues. Recently, I’ve even migrated all my DataTable structs to a plugin, and the DataTables still worked fine, though for good measure I did create StructRedirectors in the config files beforehand.

However, it’s worth stressing out that I did not use blueprint defined structs, instead C++ defined structs. BP structs should be where the danger lies.

Still, why bother with DataTables? Because of their two-way CSV capabilities. I don’t know if it’s still there, but EPIC used to provide an Excel example .xslx with ready-to-use macros to generate a CSV. You could (and still can) therefore leverage the full power of Excel formulas, data visualization, chained modifications etc… And still generate a CSV with one press of a button, which automatically updates your data table when you start the engine.

It’s very useful when you have hundreds of items with many attributes to balance, to keep track of, sort, search, visualize their stats on a graph… You will never be able to do the same with hundreds of DataAssets.

[Lupoluke]

That’s definitly my experience here.

Using data table on Struc c++ defined solved almost any issues to me.

I agree that data table should be used when you need to store multiple information you can gather later on.

[Iotti]

Most of the points that @Altrue said was right, but I’ll disagree on the point that comparing DataTables and DataAssets validity, because the person who started it just wanted to know which are the best to store static game data, which as far as I could remember, both DataTable and DataAssets can do it properly.
That said, I’d like to share a few points based on experiences I had with both DataTable and DataAssets

• DataAssets can be intuitively setted up as classes on blueprints. It helps a lot when you have lots of DataAssets but you can specify what kind of DataAsset can be setted, while on DataTable it is usually a generic type that can be hellish if you have lots of it and no proper naming convention. This changed recently (+5.2, I believe) since they have made the filtering on DataTables

• As it can be imagined based on the point above, you don’t have an intuitive way to find out what kind of data struct you have inside the Datatable. When reading you always need to parse every single line to check if it matches the struct you were expecting. It is not uncommon to see someone having issues of not reading the DataTable because the struct was not matching the one the dev were expecting

• DataTables can be parsed as .csv and json which led to big advantages when working with big teams:
  – Firstly, GD and other product related teams won’t need to install the whole engine on the same version as the devs to edit the data, just need a spreadsheet editor .
  – If your game have the need to feed any API, you can easily generate a json based on the DataTable and sent it.
  – The reverse is also valid! If you have that service live and make the request to that same API and it can be used to update your DataTable without need to reimport it everytime.

• I found that DataTables were easier to read and find an specific item but DataAssets where easier to organize

• Reading the structs on DataTables could be really ugly on blueprints, depending on the size of the struct.

• Altough the point here is for static data, it is valid to remember that DataAssets can be created at runtime.

• DataTables can be merged to generate Composite DataTables, I belive this can even be done at runtime.

• DataAssets can use instanced properties allowing you to set either instances of an specific structs or an UObject instance. This can lead to a more flexible design.

These were the cases I could remember, but the Unreal Fest presentation on 2022 they had a showcase about this:

Working with Data in Unreal Engine 5 | Unreal Fest 2022

[Philoko]

Not being able to decide if I should use Data Tables or Data Assets was the motivation for me to develop this plugin: https://www.fab.com/listings/ce66d2c8-5fbb-43a7-85d7-bd84e3ad03de

I’m still not sure what to use but at least it takes the editing part out of the equation as it allows you to edit Data Tables as well as Data Assets in a same table view.

[Altrue]

The plugin looks surprisingly nice! I’m sure it will find its niche

However, I would not recommend “use both” as a valid answer when not managing to decide anything haha. I mean, for this topic I do recommend using both in your project, but each use case should have a clear choice.

I fully agree with @lotti’s detailed answer, except for one bit about saying:

the person who started it just wanted to know which are the best to store static game data, which as far as I could remember, both DataTable and DataAssets can do it properly.

There is a clear boundary between DataTables which are for text & numbers, and Data Assets which are for binary assets & miscellaneous data that is nice to visualize (color pickers, enum dropdowns, curves, etc).

If you store a type of data in what I would call the “wrong” container, then in my view, you are not doing it “properly”.

For instance, avoiding hardcoded asset paths in your constructor, by replacing them with hardcoded asset paths in your DataTable, is clearly not “proper” in that you still have the initial issue, except that it’s obfuscated elsewhere. Whereas inside a data asset, such a reference is detected by the editor, benefits from CoreRedirectors, etc.

Let’s recap. First, mandatory criterias:

• If you want to be able to edit / read the data in a spreadsheet software => DataTables
• If you want a confortable in-editor experience when editing values (including color pickers for colors, dropdowns for enums, asset filters for asset types…) => DataAssets
• If you want to store binary assets, and the ability to use the Asset Manager to natively control how they are loaded/preloaded/unloaded => DataAssets
• If you want modders and/or internal plugins to be able to override/replace/add data (like Conan Exile does for example) => DataTables
• If you want to store text/number stuff in large quantities => DataTables
• If you want to store other kinds of assets (asset paths, vectors/colors/rotations/etc) in such huge quantities that they couldn’t possibly exist as individual data assets without lagging the editor => DataTables

If you are in one of the situations above, I don’t believe that you really have a choice.

Second, obvious choices (but not technically mandatory):

• If you don’t want your actors to use hardcoded assets paths, and you want to do it “the right way” but without too much hassle => DataAssets (it’s probably the most common use case, let’s face it :D)
• If you want to interface with some kind of API to provide data to the outside => DataTables

TL;DR The core of my argument is:
Both are useful, but in their unique & distinct ways. I suspect that the situations where their usecases truly overlap, is probably very small or nonexistent.

This is substantially different from saying “both are useful” (true, but they are not equal, therefore it is misleading) or “it doesn’t matter, I made a plugin to unify them” (you’re mostly sweeping the complexity under the rug for later).

[Philoko]

Yeah I wasn’t 100% serious when I said that “not being able to decide” was the motivation for the plugin.

I totally agree with you. Both have their use cases. This is why both are supported. Thanks for the really nice write down. I fear that not all people will be so mindful about what they actually use in practice, though. But If someone asks me in the future, I will point them to your post.

Btw. nobody said “it doesn’t matter, I made a plugin to unify them”.

[Altrue]

Haha, that pretend quote was only for educational purposes, sorry if I misrepresented too much!