Multithreaded Save Game Save/Load questions

Development Programming & Scripting
, ,
1

Hello !

I had some questions about save/load game and multithreading.

In the save system plugin we developped, we queue some UObjects operations, such a read/write/delete files and part of each operation is run in an async task to avoid hitches due to files I/O and heavy serialization.

We never had major issues with it but a few recent crashes made us investigate and we found some issues that we fixed.

But now it left us wondering if what we do on the worker thread is actually safe.

Examples of what we do :

for saving a file :

  • on Gamethread : Gather all of the data we want to save inside a USaveGame UObject, crawling into out game systems.
  • on Worker Thread :
    • Serialize the UObject into a byte array using things like FMemoryArchive, FArchiveFormatterType and FStructuredArchive.
    • Write the resulting TArray<uint8> to file using ISaveGameSystem::SaveGame()
  • on Game Thread : Finalize the operation ( callback to the UI etc )

for loading a file

  • on Worker Thread :
    • load our file into a TArray<uint8> to file using ISaveGameSystem::LoadGame()
    • Serialize the byte array using things like FMemoryArchive, FArchiveFormatterType and FStructuredArchive.
      • We create the SaveGame UObject from the right class we serialized using a FGCScopeGuard as we are on a worker thread
  • on Game Thread :
    • Apply the newly created SaveGame Object into our game systems
    • Reset the Async flag ( because it was created on a worker thread ) and mark it as garbage when we’re done. ( so it’s correctly destroyed )

So my main questions are :

  • Is the serialization process using classes like FMemoryArchive safe to use in a worker thread ?
    • we only serialize struct and basic properties in our SaveGame, no UObjects inside at all.
  • Are ISaveGameSystem::SaveGame/LoadGame safe to use on the worker thread ?
    • We are targeting PC, PS5 and XBOX so there is no unique implementation of that interface to take into account.
    • Or would it be better to use AsyncSaveGame/AsyncLoadGame on the gamethread, then using a worker thread only for the serialization process, if the previous answer say it’s safe.

Also, we block the game thread when our subsystem deinit and we have a save running operation, to make sure we don’t stop the process mid save to avoid corruption.

We also do it when there is a world change, but I think it’s just because we found online it was good practice.

  • Is waiting for the save process to be done on Deinit the right way to avoid save corruption ?
  • Is it necessary to also do it when a world changes ?

Thanks for your time and the answers !

Clément Vayer.

[Attachment Removed]

2

Hi,

Short answers, in order:

1. Serializing on a worker — Yes, with two conditions. The `USaveGame` must stay alive while the worker reads it, so keep a strong reference (e.g. a `UPROPERTY` on your subsystem) until it’s done, and the game thread must not modify it in the meantime. On load, resolve the save-game `UClass` on the game thread before you start; `LoadObject` isn’t safe from a worker.

2. `SaveGame`/`LoadGame` on a worker — Yes, on PC, PS5 and Xbox. The engine’s own `SaveGameAsync`/`LoadGameAsync` call them from a background task. However, they aren’t re-entrant, so never let two calls overlap. Run all save I/O through a single queue.

3. Engine async API — It’s the easiest way to get that guarantee: every operation goes through one pipe, and callbacks come back on the game thread. Keep your worker for serialization and hand the bytes to `ISaveGameSystem::SaveGameAsync`.

4. Waiting on Deinit — Yes, that’s right. It won’t run on a crash or when the OS kills the app, though. On PC the default save system writes the file in place, so write to a temporary slot and then swap it in (or alternate between two slots).

5. Waiting on world change — It isn’t needed for file integrity. Do make sure your completion callbacks reach UI and world objects through weak pointers, and keep `FGCScopeGuard` as short as possible: the GC forced by a level load waits for it.

Best regards,

Patrick

[Attachment Removed]

3

Thanks for your answers !

I have everything I need to make the right changes.

[Attachment Removed]