in ue5 c++ should I store data of all objects in a single file? How should I store them?

[Kart_54]

Hey there, rookie c++ leaner here, I am currently starting off making a game, with large amounts of things that can be attached the player character, split into a small number of categories (think like in Legend of Zelda Tears of the Kingdom has swords, shields and bows), except all stats stack together to create a final set of output stats that the player ends up using. I was wondering whether it is more optimal to store all categories of things in one large file, or would it be quicker to load the stats from multiple, smaller files in a folder? Also how should I store these stats? Should I be using data tables or data structs? Does it even make a difference performance difference?

I wanna prioritize performance over data space, over just being lazy and trying to get this over and done with. My knowledge in c++ is very limited (I’m either sleeping or learning c++ from YT tuts or w3schools likely when you are reading this). Any help is deeply appreciated.

[93cbbb041d664728acde881e234fed25 (Kiray)]

You may think putting everything in a single file and do 1 search to find the data that includes everything is fast. In a very small project, that may be the case. But it would be the opposite of optimized because of how CPU works in most projects.

I donno how to explain it in a simple way so I recommend look up some topics about “CPU memory cache” and “memory alignment.” The point is, CPU often waits for data than processes them. And no scalable project would put all data in 1 file.

Plan your data structures and how you store them as a programmer is fun for me.

[jwatte]

@93cbbb041d664728acde881e234fed25

it would be the opposite of optimized because of how CPU works in most projects.

This is just a plain wrong statement, for two reasons.

First, if you have 10,000 different files on disk, you have to figure out which file to read. That takes time and cache misses.
Second, even if you have 10,000 entries in a single file, that file can be arranged in some way that will accelerate access to a specific item by ID – like a hash map, or a sorted search.

Also, a single large file will absolutely load faster during game startup, than 10,000 small files.

But, in practice, the challenge for these games is almost always asset creation, not the time to load the data. And Unreal will typically bake all your little files into one bigger file when it packs up the game, anyway, so you can get the best of both worlds.

How I would structure a “character gets bonuses from objects” system, is to implement a “get bonus” interface on each of the kinds of Components you can add to your main Actor. When you equip or unequip an object, you’d simply reset your character to base stats, then iterate through all objects on the character, and call this interface (if present) to get the modifiers and apply them, and then store the calculated value as the “effective stat” value that you actually use for gameplay.

It turns out, this pattern is so common, that Unreal even has a built-in system to do this, called the Gameplay Ability System. However, because the GAS was the foundation for gameplay in the MOBA game Paragon, it’s quite fancy, and pretty hard to get going with in the beginning, especially for beginners, so I highly recommend building your own based on the pattern I describe above to get started.

Will there be a very small amount of time taken to evaluate each object when you re-calculate stats? Yes. Does that matter, at all, when it only happens when the player changes equipment? No. You won’t be able to actually measure the difference in frame rate. And, once pre-calculated, the “effective stat” value variables are fast to access and use.

[93cbbb041d664728acde881e234fed25 (Kiray)]

Perhaps because I didn’t explain much but only bring up topics. I assume the post is going to follow a common way of storing data using Unreal Engine’s DataTable.

Unreal will typically bake all your little files into one bigger file when it packs up the game

As jwatte said, Unreal Engine already bake things in build. Files would already be condensed for you. You wouldn’t want to put every data in a single DataTable would you?

I was wondering whether it is more optimal to store all categories of things in one large file, or would it be quicker to load the stats from multiple, smaller files in a folder?

If DataTable is implemented so that a single structure would hold all the possible variables like stats/asset_filepath/texture/material/mesh/icon… It would not be good. Would a weapon use variables from armor structure?

It is bad to pass around for all the copying structure do. When an array/map/list of these unpredictable structures are in memory, that’s why I bring up the topic of “CPU memory cache” and “memory alignment.”(hope you understand where I am coming from)

Back to the post.

I am currently starting off making a game, with large amounts of things that can be attached the player character, split into a small number of categories

I’d say split your data structure as how you’d categories them. And separate assets from data. Like you have Weapon/Armor data with their own structure. But have GearAsset data with common variables like Mesh/Material/Texture in the structure. Add an assetID to your Weapon/Armor structure so that they can find it in GearAsset. Load these assets separately and asynchronously if possible.

Should I be using data tables or data structs? Does it even make a difference performance difference?

DataTable is a kind of data structure. It has more going on as it supports/follows UPROPERTY(). Like if you have a DataTable with references to some texture assets, the assets would be loaded into memory every time you use that DataTable. That would be a performance hit unless you use a soft reference.