I’ve been getting back into UE5 BPs recently; I’ve been reading a lot about Data Assets, and I’m worried about my current Inventory System.
Here is how it works for context:
The Inventory is a component, attached to the PlayerCharacter. Inside the component is the logic and data for the inventory.
The various data is stored in the Inventory as an arrays of structs, one for Items (ItemStruct) and one for Weapons (WeaponStruct), as weapons have more information needed (like firemode).
PickUps (for both Weapons and Items) are Objects containing a Struct, said Struct is composed of the RowName ID and the non-static Data (content, durability, or firemode, ammo in the case of weapons)
Upon picking up an item, the InventoryComponent gets the RowName ID and gets the Row from a DataTable (DT_CommonData) containing all Static Data shared by everything the inventory can fit (information like the name, the weight value, the item size and volume it takes in the inventory). Using that static data from the DataTable, the InventoryComponent checks where and if the item can fit, and if the function returns success, the Data from the Struct in the PickUp is added in the corresponding Array, and if it’s a success, the PickUp is destroyed.
Dropping an item is the reverse operation; spawning the right PickUp, specified in the DataTable, setting the Data, and removing the index from the Array in the InventoryComponent.
When an Item or Weapon is ‘equipped’, the InventoryComponent gets the DataTableRow from the ID in the appropriate DT (DT_ItemData for Items, DT_WeaponData for Weapons) which contains the information to initialize the Item through an Enum; for instance “is the Item a container/Food/Tool?”. Everytime the player equips an Item or Weapon this process happens, fetching the static data from the matching DT.
After reading a lot about how Data Tables and Data Assets work, I’m under the impression that my Inventory System isn’t all that great performance wise, and I’ve failed to find a proper answer, let alone a solution.
I’d be grateful for any clues, advice or information regarding these questions I have:
I have read that everytime a DataTable is called, the whole is loaded; knowing that my DataTables will contain hundreds of Rows at the end of the project, it sounds harmful to performances; should I consider another solution for getting and storing my static data? Are Soft References a serious way to address this concern?
I’ve read that Data Assets are great for storing non-static data, should I abandon Structs as a means to store my non-static data for items and weapons?
I want to make my project scalable and unburden my future self with complete reworks of systems, but I feel I don’t understand how Structs, Data Tables and Data Assets work quite yet!
To be honest, I think your is the best approach there is. Yes, the whole data table is loaded first time it is accessed but I see this as a perk, not a problem. You can have the names and icons of all items readily available throughout your game. The table itself should be manageable even if you have thousands of entries. (unless the structs are truly gigantic)
That being said there are several things you can do to optimize that.
Make sure your structs are light and with fixed size.
Store copies of those lightweight structs both in the inventory and in the pickup (instead of just the id) This will minimize DataTable look-ups and traversal. (although finding an item by id in a sorted array is lightning fast)
Soft references are useful to hold data that does not conform to the “light-fixed size” struct. You should keep the easy and useful parts of your items in the DataTable and large and unique parts (meshes, animations, 4k textures) as soft reference to load only when needed (lets say when equipped).
Data Assets are useful in the following scenarios:
You work with a large team and everyone working in a single DataTable is just not manageable.
You have smaller number of large objects.
You want to control how the assets are packaged.
You want to stream your data because it is too large.
P.S.
I forgot to mention that this is not a clear cut case and there is a lot of opinion involved. You can read all day about DataTable vs DataAssets but, in the end of the day, you can use both of them with very similar effect.
this is kind of false, yes you can modify the data but you’re doing it to the baseclass which effects all instances. ie if you changed durability all items would change.
i personally prefer DataAssets, the big advantage is they support inheritance so you can have the one item class with subclasses for weapons, armor etc.
that said dont change your whole project if its already working
Yes, if you use any one row in a data table, then the entire table is loaded.
Assuming that you use soft asset/class references for the “what is this thing?” fields, then that is unlikely to even show up in a profiler.
A single mesh for a single flower will probably use a lot more memory than this table.
I would recommend against mutating Data Assets. Store mutable data on an actor or component of some sort, or as a struct in some container that in turn is stored on a component or actor.
And, yes, soft references is the way to improve memory behavior of large systems. They don’t load assets into memory until you resolve them. Of course, this means that you may get a small hitch/glitch when you first use some particular item during the game session. If your game is a single-person development, you will probably not be able to develop enough content that a gaming computer couldn’t just keep it all in RAM, so you might be better off just loading everything and calling it good.
The only answer to “is bad for performance?” comes from a profiler. Run the game, on the target hardware. Measure its performance. If performance is good, the thing you’re doing isn’t bad for performance.
So far the struct for the DTCommonData contains roughly 20 variables, half being Integers and Booleans, but some being 2D Textures (icons), Text and Enums, is that veering towards a non-light struct? I’m unsure as to what constitute lightness in a Struct, but I’d assume it’s “few and simple variables”?
Regarding Struct Copies; the way my WeaponComponent works is on Equipping/initialization of a Weapon the data is fetched from the appropriate DT and then stored in the Component, because I had a bad feeling about getting the DT row everytime the Weapon was used to get its Base Damage, there will be fast firing weapons, so I thought it’d be better to save these values directly.
I could get that data from the PickUp and store it in the Inventory upon succesful Pickup if it’s better performance wise?
I think I misunderstood what you meant when you mentioned “keeping the easy and useful parts of items…”; should I remove the meshes and animations from the DT? Would I move them to another DT or is there another way to know exactly which mesh/animation an item would need without a DT?
Thank you a lot for your insight, you have already lifted a huge weight off of my shoulders, I am grateful!
Would you use Data Assets for, let’s say, Sub-Types of Weapons Logic? For instance a Baseclass “Gun” with a few subclasses like “pistol”, “automatic” and “shotgun”?
I am interested in learning new tools, so even if it’s not the answer for the InventorySystem, it could still make life easier for another part of the project!
Thank you a lot for your input, I really appreciate it!
its possible, the answer really lies in what your game needs.
too much inheritance can get problematic too, i’d probably just have a baseclass ‘item’ with subclass ‘rangedweapon’
but really for indie games its not too important as long as you know what you’re doing.
I’m unsure as to what constitute lightness in a Struct, but I’d assume it’s “few and simple variables”?
The “best” structs contain plain data only with fixed size. The main goal is to align the data in memory and prevent cache misses. I’m don’t feel competent enough so I won’t even try to explain these topics. Here is some info if you want to learn about that:
I could get that data from the PickUp and store it in the Inventory upon succesful Pickup if it’s better performance wise?
I doubt you will see any difference if you don’t make it constantly (on tick) but having fewer redirections is always faster. The downside is you fill your memory with redundant data but from the topics above you can see that data locality is actually more important.
should I remove the meshes and animations from the DT?
They are probably stored as references there anyway but yes. I would make them into an object/actor and leave a soft reference in the DataTable to it. Next I’d load the soft reference only when the object is taken by the player. These could be soft references to Data Asstes even
I’m sure there’s some kind of micro-benchmark where this matters.
Also, if you’re building a massive multiplayer game with a thousand people all firing automatic weapons at the same time, this may matter.
However, it feels to me as if you’re way over-optimizing something that probably doesn’t matter, and doing it way before it’s time to worry about this level of optimization.
This is the kind of optimization that was meant when Knuth said “premature optimization is the root of all evil.”
It’s true that choosing the correct algorithm is important! Scanning through every actor every frame to find something, when you could just have cached a reference to that something, is a bad algorithm, and should probably never be used in the first place. Selecting the right algorithm, is fine design work, that you can and should do early. But, optimizing struct sizes for cache behavior, only matters if you have literally thousands of them, and scan them all every frame.
Optimizing the struct layout for each particle in a particle system? Abso-freaking-lutely!
Optimizing the layout of the inner loop of a DSP function for a procedurally generated sound? Probably a good idea!
Shaving one cache miss off of something that maybe happens five times a second? Nobody will notice the difference, on any machine that is at all capable of running Unreal Engine. Even if you fail to do this optimization ONE HUNDRED DIFFERENT TIMES and it happens EVERY FRAME, you won’t notice one hundred cache misses per frame. The cost of one cache miss is 0.08 microseconds. The cost of one hundred cache misses is 8 microseconds. 8 microseconds at 60 fps is 0.03 fps difference.
As long as you choose a path and stick to it, you’ll be fine. The two paths are:
You’re a one man band. You will never be able to develop enough content that a modern computer will worry about it. Load it all in RAM and call it good! Ship a game, be happy!
You’re extremely worried about initial load time, so much so that you change all your asset references (icons, animations, sounds, …) to be soft references, and load them on demand. Your game loads slightly faster! Ship a game, be happy!
Also, the way that sub-classes are done in Unreal is generally by creating blueprints of the subclass. Each blueprint has the customized ammo/fire-rate/sounds/meshes/icons. An actor blueprint is a class, quite physically. Thus, you develop all your guns as blueprints that derive from your “Gun” base class. (Which may also be a blueprint, or a C++ class.)
Then, in inventory, you have a struct that has a soft class reference to the blueprint of the item. The inventory struct should also contain a soft asset reference to the icon to show in inventory, and the user visible name as string. You don’t need to add parameters related to the gun behavior, unless you want to show all of that in the inventory, too.
