I just wanted to post a significant update which improves on serializing game data. I spent the last week trying to simplify and reduce my save game system so that it’s easier to use, uses less data, and is more elegant in its design.
**Changes:
- I don’t use an object manager anymore in the game state.
- I keep track of save file versions for build compatibility
- I support saving pointers and lists of pointers
- You’re not required to “pollute” your game objects with savegame meta data**
I will share all of my code files, and then talk about design and implementation details. I have designed these code segments to be project independent, so I have this working in a blank project with starter content. Each of the following are source code files attached to this post (too much to copy and paste).
CustomGameState.h (1.95 KB)
CustomGameState.cpp (5.52 KB)
ISaveable.h (2.05 KB)
ISaveable.cpp (947 Bytes)
TestSaveObj.h (1.06 KB)
TestSaveObj.cpp (3.05 KB)
MySaveGame.h (540 Bytes)
MySaveGame.cpp (112 Bytes)
Important Note: You will need to use C++ in your project in order to use this. I tried really hard to minimize the amount of code required and use the editor / blueprints as much as possible for ease of use – for now, you’ll still need to write C++.
In this version, I spent a lot of time trying to eliminate and reduce my save system as much as possible. In my previous design, I was maintaining an object database during game play. The idea was that any time you spawn an object, you have to tell the game state about it and it had an object manager on the backend which would assign that object a unique ID from a pool of integers, and then inject the object into a hash table (TMap). When it came time to save the game state, I just saved the contents of the object database. Since I tracked every object and it had a unique ID associated with it, I could convert object references to integers, which would be useful for serializing pointers. There were a lot of problems with this design approach:
-
It creates a lot of variable pollution in the game state. I had to have a unique ID pool and the supporting code infrastructure to manage access to it. I also maintained a game object database.
-
If an object isn’t registered with the game state upon spawning, it won’t get saved. It requires users to make sure that they add the object to the game state so that it can be tracked and managed. It’s a potential source for human error (forgetting!) and also adds a lot of extra code bloat. Any time an object gets spawned, it has to get the game state, cast it to the right game state, and add itself. Yuck.
-
It consumes memory.
-
The design was too tightly coupled.
Usage:
The updated system completely gets rid of all of this. The only data structure I still have is an array of object records. This is empty for 99.9% of the game and only gets populated during saving and loading, and after saving or loading is complete, it gets emptied. It’s “transient”, but not. On the blueprint side, I expose two functions: SaveGameState() and LoadGameState(). From the blueprint designer side, a user just needs to worry about calling these two functions and it will “magically” work.
If you look at “CustomGameState.cpp”, you can see the implementations for each of these functions. You’ll have to touch each of these functions lightly. Within the SaveGameState(), you’ll have to create a list of function calls for SaveActors(), with the class of actors to save. You’ll probably also want to modify the SaveGameInstance values to modify the savegame slot name and profile name. One thing I didn’t implement is the current level the game is saved within, so you may want to do that if you have multiple save games with different levels.
If you look at the “SaveActors()” method, you’ll see that it gets a list of all actors of that class type and then it tries to make an interface call on them. If the actor implements the interface, its “SaveToRecord()” method is called. This is super important!!! Every actor you want to save, will have to implement this interface. The reason is because actors may contain pointers or lists of pointers, and I can’t serialize a pointer directly, so the actor will have to implement a schema for converting pointers to something that can be serialized (more on this below).
The “LoadGameState()” method is 99% self contained. The only thing you may want to modify is the name of the savegame slot which gets loaded. This function is where most of the magic happens and I think it’s very elegant. This is where I implement a two pass deserialization scheme. In the first pass, I load all of the actors from the saved file and then spawn them in the world. The only thing which is missing is any pointer links. I can’t relink pointers until every actor is spawned because a spawned actor could be pointing to another actor which hasn’t been spawned yet – so, I need a second pass. In the second pass, I go through every spawned object and call its “Relink Pointers” method and pass it a list of all spawned objects.
Serializing Pointers and lists of pointers in an actor:
Pointers are hard. Let’s review pointers briefly. A pointer is a variable which contains an address to memory, which contains another variable type. This usually contains a memory address which is located on the heap, and usually is an instanced object. Pointers themselves are variables with an address, so you can also have pointers to pointers. So, a pointer value could be a memory address such as 0x0552FC30, which corresponds to the memory address of an object. The critical thing to note is that every time you run your game, without changing any code, the address of the object being pointed to will be different!! So, if you are dumb, you will serialize the pointed to memory address and think that it contains a valid reference to the same object. Completely wrong! We should assume that the memory address of an object is random, so serializing random numbers is fruitless. The technique I came up with (and turns out to be sort of common) is to assign each object a unique object ID (sometimes called an OID in other reference materials), and then when you find a pointer which points to an object, instead of serializing the pointer, you serialize the Object ID. When you restore a pointer, you have a unique integer which corresponds to an object in memory, so you just use the OID to lookup the memory address of an object and then relink the pointer.
An object will often have more than one pointer. It could have dozens of pointers, and it could have lists of pointers. The key thing to note about lists (arrays) is that the length can be variable. If you look at my “FSaveDataRecord” struct located in “ISaveable.h”, you can see: TArray<uint32> PointerList;
As far as the save file is concerned, it’s just a list of integers which it saves and it neither knows nor cares about what those integers mean. You get to decide what those integers represent, and that decision gets made per interface call. If you have an actor which contains five fixed pointers, you can add five integers representing the OID of each pointer. If you have a mix of pointers and arrays of pointers, you can add extra numbers to the array to act as meta data. In my case, I save the array count of a pointer list, followed by a list of OID’s. When you load the pointers back in, you know how many you are going to load before moving onto the next item in the array. If you use this meta data schema, take great precaution and care to test your implementation for correctness – I find this is a high source for a lot of human error.
If you look at TestSaveObj.cpp (3.05 KB) and my implementation of “SaveToRecord()” interface call, you can see examples illustrating each of these techniques.
I made an important discovery as well: Every single object in Unreal Engine has an assigned unique ID. It’s a uint32 stored in UObjectBase. To access it, just call “GetUniqueID” on any object in UE4. I think they use an ID pool on the backend so unique ID’s are recycled – they aren’t GUIDs. For the purposes of saving and loading games, this is perfect and helps avoid variable pollution. This is what I use for my OID and allowed me to delete my own implementation of an ID pool in game state.
Possible Improvements:
Most likely, out of my own ignorance, I don’t know how Unreal Engine serializes pointer references. I know they do it somewhere, and it’s probably really easy and elegant and probably happens automatically. For now, I’m stuck with implementing a two pass loading scheme and forcing users to implement a save/load interface. There’s gotta be a better way to do this such that it’s transparent to the user. I’m sure it’s a solved problem for a lot of people and teams, I just don’t know how other people serialize their pointers so I’m stuck with my janky solution.* If you have a better way, PLEASE COMMENT!*
If you’re a glutton for punishment, have a lot of time on your hands, or improving serialization is your focus for the next few months, you could move away from the FObjectAndNameAsStringProxyArchive for the savegame file. If you look at the resulting savegame file in a hex editor, you’ll see that it lists objects by strings followed by values. You could implement a raw binary format where you minimize the amount of metadata which gets serialized. This could potentially shrink the size of your savegame files by some unknown percentage, which is probably significant. For most people, this would be a waste of time and a source of bugs, but if you have a strong command of serialization, binary data manipulation, and data compression, you could potentially turn a 500kb savegame file to 5kb? This might open up some new capabilities when it comes to multiplayer games and game state synchronization. If someone joins a game in progress, you could serialize the entire game state into a tiny data packet and send it to them. Alternatively, if a multiplayer client gets disconnected from a game and then reconnects at a later time, you can gracefully resume the game by sending them a game state data packet. Again, this might be a waste of time depending on the size of your game state data file – improving 500kb to 5kb is nothing, but improving 250mb to 25mb might be valuable.
If I come up with any further improvements or learn something important, I’ll add another comment. For now, this should be a really good starting point for anyone who wants to implement save games which contain level state information.