Which would be the most appropriate Unreal object (or whatever) to use for attaching my custom data storage subsystem?
Is the GameInstance appropriate place or is there any higher level available from C++ code? I have read this:
but I’m not sure if this is the best solution, considering my requirements and implementation plans (see the details below).
The long story.
For my custom hobby multiplayer simulation project, I’m going to implement a persistent storage subsystem (for now I’m aiming at MySQL but I hope to implement it in abstract-ish manner so I could attach Redis or MongoDB or whatever later on).
I am mostly experienced with web app architecture but I’ve read lots of articles about game server architecture as well. So, now I’m feeling ready to get my hands dirty with some coding. After reading UE4 documentation and C++ beginner and plugin developer guide, I have only one question - where to attach my code?
Hmm… maybe UE4 already supports custom data stores or is on its way to implement that…
Some requirements and implementation ideas which come into my mind:
I don’t want to recompile entire UE4 engine, I want just to create a plugin which could be used in any UE4 game.
the storage subsystem should exist as a single instance throughout entire game (even when switching between modes in UE4 editor / play / dedicated server etc.)
the storage subsystem should have access to the global game configuration when it gets loaded - I’ll need the SQL server host, port, db name, user, password
the storage subsystem should receive Tick events or should be able to run in a background thread. I don’t want to write to the storage on every Actor property change. Instead, I will use some dirty checking on all actors which have my custom Storable component to see if it has some pending writes and then write some of the pending updates at the right moments (will try different strategies - check for idle CPU or just use some timeout and distribute updates in batches to avoid flushing them at once and causing game jitter).
the storage subsystem should have access to all the Actors in all the sublevels of the world root level (to support level streaming)
later I might get fancy and implement transaction support to ensure that the actors which have Storable component marked with the same transaction ID get actually stored during single SQL transaction.
Also, I haven’t yet looked into UE4 multithreading facilities - in case if my storage subsystem runs in a dedicated thread I’ll have to use some lightweight locking mechanism on my Storable component to avoid race conditions… but these are just ideas and problems to solve as I go. At first, I need to get the right starting point.
You can write a plugin in C++ and add that to the engine, that will work fine. That plugin can spawn its own thread (with all normal thread caveats) if it needs to. You can talk to that plugin using Blueprints, if you want.
Talking directly to a database server from a game client is a really bad idea, for both performance and security reasons. Instead, I suggest that you implement the storage code only on the server. Clients will receive events from the server. The server will receive events from the clients. The server will do the necessary I/O to the custom database. Or, even better, the server may talk through some network protocol (Thrift, HTTP, etc) to an application server, which talks to some data store.
Are you sure that the storage needs to actively reach into the engine and muck around with actors? Wouldn’t it be better to fire some events and let the game code read out the information it needs, and go from there?
Start a C++ project. Create a new class from the Unreal Editor – perhaps as a Blueprint Function Library. Expose some functions on that plugin to the engine. You might want to derive from the existing database support classes. Call those functions from the game, only on the server (in response to “run on server” events.) You can go from there!
If you want to move on from just having your own C++ project (which uses the engine as a library) and provide a true, separable, C++ Plugin, you can check out the instructions here:
It’s often possible to take a previously developed class and move it to a plugin structure, so you don’t necessarily have to make that decision right away.
Thanks, jwatte, for ideas - yes, sending events would be much better approach than direct Actor access, I definitely will try to implement sending events (or use a thread-safe queue) as somewhat abstract interface on my Storable component - it will send events about pending data changes, and the storage system will decide what to do next - send the dirty data to DB right away, collect and send later in batches, wait for CPU idle etc, based on config settings or whatever I’ll be able to actually implement.
Yes, my intention is to use it for the UE4 server side, but I imagine that someone might want to play with it when not using client-server UE4 architecture - just for experimental purposes or some quick&dirty projects. Therefore I won’t lock the use of the plugin for server side only.
When using it on the UE4 server side, I might get it done without using custom protocols etc. if I manage to use UE4 network logic, which, judging from some tutorials, seems to work fine for syncing Actor changes and controlling them on the server.
Currently I’m just not sure which would be the best “attachment point” for my plugin to the engine. Which UE4 class or event handler should I use to init the storage system and open the connection? How to expose the storage system singleton to UE4 Actors through my custom Storable component and also through some other interface for cases when I want to access the storage system directly from UE4 Actors and blueprints without any specific Storable component?
As there will be Actors stored in database, how should I spawn them based on database records? It seems, at one point (which UE4 event?) I will need to ask the storage system to read all the Actor records and spawn them into the game world, taking into account that UObjects are not thread safe, and thus I cannot perform this task in a background thread.
I guess, there might be multiple choices, but I just want to get it right to avoid trials and errors and complete redesign.
GameMode might work; it runs server-side only, and determines what kind of game is being played.
Or you can do it on demand whenever some blueprint, script, or actor asks for the necessary data.
If your server-side code needs to spawn actors in response to the game starting, Event Begin Play in the GameMode might be a good place.