Alright guys, I’m having an issue. I am trying to build entirely using blueprints(Can learn C++ but would rather not). This idea is going to be similar to an MMORPG, its going to be highly instanced in that most of the actual gameplay will take place in a location that has a maximum player count of 16 or 20. I need to develop some type of backend that interacts with SQL to store player data and item stats. I have a pretty good understanding of how blueprints work and coding is not a foreign object to me either, however my skill set so far encompasses python and only python. Could you set me down the pathway to achieve this?
Your client or dedicated server should not talk to a SQL server directly. Instead, you should build a backend server using HTTP (PHP/Python/Rails/.NET/etc.) or even something talking to a server in real time over a TCP connection. That backend server should talk to the database. If you choose the HTTP path and DON’T want to learn C++, there is a plugin called VaRest which allows you to deal with JSON in Blueprint.
I’m having trouble constructing my basic model for interaction. There is also no clear documentation of what functions are available to a user of that VaRest plugin. Some helpful souls over in the #Python channel over at freenode IRC said that to avoid conflicts my best bet would be to look into a JSON database and have my dedicated server talk to and store directly to it. I have no clue how the structure of a JSON database looks and how similar to SQL it is when pulling or storing records into it. They have stated that if I were to write a python program to listen to the JSON that it would require a massive amount of resources to interact with my suggested playerbase volume and there would be the risk of said JSON file hitting its max size before the python program saw the information and was able to store/confirm storage and remove it from the JSON to make room for more data. I am drowning here.
I think they were getting at using something like MongoDB which is designed as a document database and goes hand in hand with JSON and SQL is still fine too. Document databases and relational databases have pluses and minuses depending on what you want to achieve. You can also write JSON directly to a file on the hard disk the server is running on. It really depends what you want to do.
If your game is not going to be persistent between servers then having flat JSON files could work and should be fairly lightweight. You can also directly talk to a NoSQL or SQL database in this case too and maybe possible with persistence if your dedicated server binaries are going to be private and all game servers are to be running on your own servers. However, if your dedicated server files are going to public and you do want persistence between servers you will want it to talk to a backend server first before hitting the database as the backend would authenticate connections and sanitise the input data.
I do want my game to be persistent between servers, but according to MongoDB’s site JSON files are capped at 16mb and I’m more than positive with all of the data I need to store I will exceed that. Unless MongoDB can chain the JSON files together so say, player logs in game needs to pull his character model info, character name, equipment info, health, abilities. Thats a lot of data per character to store/recall and it all has to be stored server side. So if I can store those things and MongoDB can sort out those requests. I’ve never dealt with an issue like this, it’s a tad overwhelming due to the fact that JSON files are outside my scope of knowledge so I don’t know alot about how to interact with them. As well as UE4 is new to me so learning how I can get data OUT of the dedicated server has been a venture as well.
Where does it say that on MongoDB’s site? Perhaps it’s a 16MB limit per-record and a record for an individual player is not going to take up anywhere near that much space.
Yes its 16MB per record, but I still fail to see how I’m going accomplish my goal of storing/retrieving player information with VaRest/MongoDB. The two would require a bridge of some sort to create the interaction between server/DB. Why would I not want my dedicated server to communicate directly to a DB again? If you could walk me through the process of creating a .cpp file that held my SQL connection info, and then creating a blueprint that I would elevate to a server execution only blueprint. I could cause the server to initiate a connection to the SQL server and then walk me through creating some basic blueprints for pulling data from a server based on a 1-3 variables, I could very easily accomplish my goals with that. If its possible with VaRest/MongoDB and I’m just missing the point please explain it to me, I probably am just missing the capability. I understand they both work off of JSON files but MongoDB’s role as far as I understand is, he just stores JSON files and then just like other database applications you can query to pull specific json files. So while the VaRest would be able to write JSON files that a middleman application could have MongoDB store, we have no way of querying MongoDB to pull the specific json file and then have VaRest switch to that particular JSON file and deposit the information needed. VaRest is a great plugin but it lacks the power of accessing/using a large amount of data. It is limited to a externally updated JSON file. While it can make changes to that JSON file it would require an extreme amount of effort to design and utilize an application to work around VaRests shortcomings. I really appreciate how quickly you’ve been replying to me and how helpful you’ve been to me. I know my knowledge is quite limited and I can’t explain things very well. Hopefully you will continue to work with me until we approach a resolution!
You don’t want the game talking directly to a database (especially one you don’t control) for security reasons. There are plenty of things that can go wrong when you do that, just ask Edmund McMillen.
A RESTful HTTP API that serves JSON is how you’d go about it. There are many solutions for pretty much every web development platform on the planet that’ll allow you to achieve that.
If you Include Http.h you can use HTTPRequests. You only need to write php scripts that interact with your Database.
You can POST requests with over HTTP inside your Project. The POST Data can be a JSON String.
In your php Script you can echo the Query result as a JSON too.
That’s how i made a login system. I also made a script that refreshes sessions every minute and is called by a Timer inside my level Blueprint.
But i moved over to Steamworks, since i don’t need a complete login system and servers to save inventorys. I only use the local
savestates for saving maps and inventorys so i haven’t finished the HTTPRequest thing.
But i guess you could work with it.
So your suggestion in short, go learn about RESTful HTTP API’s that serve JSON and use PHP or Pythong to build your backend. So here’s what I’ve got for my model: Anything involving changing a skill or piece of gear is a direct connection between client and REST enabled webserver. While anything involving movement, combat or environment interaction takes place between webserver and dedicated server and eventually client but all data exchanged for combat calculations is between webserver and dedicated server. Would that work?
I’m not sure about the idea of the web server being responsible for combat calculations, is there a reason your game’s dedicated server couldn’t do that? Storing stats and inventory at intervals or on certain events (eg. disconnect) is fairly lightweight but for something that will be done constantly in-game HTTP might not be the best way to go. If that’s the case you’ll have to look at using something more real time by having a central server that your game’s dedicated server connects to that has a persistent connection and can send and receive data with low overhead. If that’s a route you want to take then you’ll have to look at sockets programming and using serialisation libraries like Protobuf and Apache Thrift.
The other person who commented earlier said
Okay, could you point me in the right direction of a specific language and some source of information to get started?
Yes,
i hope i find everything i needed for this. First you need to add Http to your Dependency.
You can do this inside your Build file. “YOURPROJECT.Build.cs” and yes, this file is C#, but that’s not that hard.
You should find a line like this:
PublicDependencyModuleNames.AddRange(new string] { "Core", "CoreUObject", "Engine", "InputCore", "Http" });
But without the “Http”. Just add it.
Now you create a simple Code for you project. For example with the Actor Parent class.
Now include the “Http.h” header file. I found it here:
#include "Runtime/Online/HTTP/Public/Http.h"
There are possibilities to shorten the path, but i’m not that skilled. You can probably look up Ramas SteamNetwork Guide to
copy the lines for shortening the Include paths.
Now you need to create at least 2 functions inside your Header file (the one of the new Class you have created).
It should be structured like this:
void LoginRequest(FString PostData);
void OnLoginRequestComplete(FHttpRequestPtr Request, FHttpResponsePtr Response, bool bWasSuccessful);
For each different request function, you would need to make an additional 2 functions.
The first one is the request, the second one is a delegate that is called after your request (at least i hope that it’s like that,
but you will still understand what i mean).
Now inside your .cpp file, you would need to include Json.h too, to use Json string functions.
The first function “LoginRequest(FString PostData)” should look something like this:
void AHTTP_LoginAndID::LoginRequest(FString PostData)
{
TSharedRef<IHttpRequest> HttpRequest = FHttpModule::Get().CreateRequest();
HttpRequest->SetHeader(TEXT("Content-Type"), TEXT("application/json"));
HttpRequest->SetURL(TEXT("http://127.0.0.1/gamelogin.php"));
HttpRequest->SetVerb(TEXT("POST"));
HttpRequest->SetContentAsString(PostData);
HttpRequest->OnProcessRequestComplete().BindUObject(this, &AHTTP_LoginAndID::OnLoginRequestComplete);
if (!HttpRequest->ProcessRequest())
{
GEngine->AddOnScreenDebugMessage(1, 600.0f, FColor::Green, TEXT("Request failed at start!"));
}
}
The URL, for example, is the path to your Webserver and the matching php script.
The Method you use is “POST” because you POST information to the script.
The BinUObject is the Delegate. It’s the second function you created. You bind it to the Request event.
The IF Clause is something i made. You don’t have to use the AddOnScreenDebugMessage.
Your FString is the Data you want to sent. You can use Json Strings here.
The Delegate looks something like this:
void AHTTP_LoginAndID::OnLoginRequestComplete(FHttpRequestPtr Request, FHttpResponsePtr Response, bool bWasSuccessful)
{
ASurvivalCharacter* MyCharacter;
FString MessageBody = "";
if (!Response.IsValid())
{
MessageBody = "{\"success\":\"Error: Unable to process HTTP Request!\"}";
GEngine->AddOnScreenDebugMessage(1, 600.0f, FColor::Green, MessageBody);
}
else if (EHttpResponseCodes::IsOk(Response->GetResponseCode()))
{
MessageBody = Response->GetContentAsString();
GEngine->AddOnScreenDebugMessage(1, 600.0f, FColor::Green, MessageBody);
// Cast the found AActor to ASurvivalCharacter
MyCharacter = (ASurvivalCharacter*)UGameplayStatics::GetPlayerCharacter(GetWorld(), 0);
MyCharacter->AccountID = FCString::Atoi(*MessageBody);
/* JSON VERSION
FString JsonRaw = Response->GetContentAsString();
TSharedPtr<FJsonObject> JsonParsed;
TSharedRef<TJsonReader<TCHAR>> JsonReader = TJsonReaderFactory<TCHAR>::Create(JsonRaw);
if (FJsonSerializer::Deserialize(JsonReader, JsonParsed))
{
MessageBody = JsonParsed->GetStringField("id");
}
*/
}
else
{
MessageBody = FString::Printf(TEXT("{\"success\":\"HTTP Error: %d\"}"), Response->GetResponseCode());
}
}
This is just copied from my test project, so you would need to change the ClassName and some other things.
What i’m doing here is the following:
I check the Response. First if it is valid and second if the ResponseCode is ok.
If both is ok, than we can use the Response. “GetContentAsString”, for example,
returns the String that you send back as an echo inside your php script.
In my case, it is a unique userid. I display it to check if everything is working and save it to my character.
The commented section with “JSON VERSION” as title was just a test to deserialize a Json string that was echod by a php script.
You can make multiple functions like this, to Request different things and make them BlueprintCallable to use them in BPs
(: Hope that helps.
Wow, mysql_query, really? You know that’s been deprecated for the longest time and you should use mysqli or PDO instead. Is that also vulnerable to SQL injection too? Also you shouldn’t use local files or Steam’s cloud saving functionality to save persistent data about a player as that can easily be cheated.
That point me in the right direction was directed at sswires. Can I use Python to write a winsocket backend? What do I need to do where should I start?
So according to my research I can use Python for the socket server and make it perform all of the tasks I need. Would that take the place of my dedicated server? Could I use it to do just what I need and continue to allow the Dedicated Server to handle the rest? So basically I would need someone to help me get started as to helping me put the Cpp in that allows the game client to talk to the python socket server and then help me write some blueprints for being able to send out certain variables and whatnot. From there I should be able to man the helm without a GPS if you catch my drift.
Python can be yeah and there’s plenty on Google about creating a TCP server in just about any language. You just need to make sure your game and talk to it and vice versa. You’ll also want it so that your TCP server runs on multiple threads so that it can deal with multiple clients sending data simultaneously. I’ve only done sockets programming in C# and a little in C++ using Winsock. For Unreal, I imagine there will be a TCP sockets abstraction layer (for cross-platform) in there somewhere.
Yeah I found lots for using python with this framework 20.17. SocketServer — A framework for network servers — Python 2.7.18 documentation seems to do exactly what I need. All I need to do now is get Unreal to spit out some data so I can see what it looks like so I can build the server from there. Did you see my other post? Right above your last?
It wouldn’t be in place of your dedicated server as your dedicated server would handle the game world itself and interacting with other players. Your Python server would be responsible for doing any persistence such as storing character data or any other data you want to store or calculate centrally rather than on the game server itself. Though I do just recommend having it store player/world data and actually performing actions (such as combat) on the game’s dedicated server.
Yeah now that the interaction between DB and Dedicated server has been simplified so much, I can have the clients log in and whatnot through the socket server and then the socket server gives them their character data, then the client sends its character data to the dedicated server and the server checks it for authenticity with the socket server. After authenticity check server calculates the values that will be necessary for combat interactions and hands them back to the client, probably have these values refreshed every 5 minutes or so. When combat occurs i.e. ability gets used spell cast whatever, player says I’m hitting dudebro heres my info hands the relevant information to the dedicated server, server handles the calculations for combat and the combat resolution is passed back to involved clients. Sound good? You’ve been so friggin helpful. I just need one more thing, You mentioned TCP Socket abstraction layer, does that mean that the application already has the capability of starting up a data stream and that blueprints are already available to use for passing data to a socket server? If not, where would I get some information, neigh an example of how I would go about adding a .cpp to my project to handle this data stream. And make some blueprints for handling data. Would only need 8-9 blueprints for what I can think of right now.