Thanks! I’ll give Eclipse a look.
I hope you will not be offended, but I’m going to take this offer at face value. 
If it’s not too much trouble, could you provide some guidance on learning how to load your sources into Eclipse to create a project that runs on its own? Any links or advice would be much appreciated. At the moment, I am studying very basic tutorials and have added the files to a project but these do not suggest my next step.
Also, might you offer any suggestions on a specific usage of the plugin and the server for connection to a MySQL database? My intention is to use the sockets plugin to load and save data through the java server. This data will be of the sorts used by Inventory, Chat, Stats, and etc systems. I would like a single, unified method for handling data. Do you see any problems with this approach?
I am studying tutorials on Java - MySQL connection and the necessary coding seems to be within my limited abilities.
I am a programmer of sorts (I was trained to program in Pascal) and do have a satisfactory understanding of general concepts, but Java is new territory for me and learning a new IDE and methodology has always been a painful process.
Thanks in advance!
Sure, I’ll put a tutorial in Eclipse together, that explains things mainly through screenshots.
It will take me some time to do this though, so don’t expect this to be done today 
I can imagine that for most games the 5 MB limit is more then enough ! I totally agree on that !
For my project I will use an update server and an account server and I’m quit confident that the 5 MB/frame limit will be more then enough there !
But, for instance in my case, on the game side, UE is more like a really advanced UI and the real heavy logic, AI, data connections, etc are all done in the Java-part as I use my own framework in Java for this. It is also easier to control than the rate of updates Epic follows and I have no influence/control on at all. Anyway, I like to avoid problems like with SQLite, version X of UE supported it (not officially) and the next versions didn’t. The datatables (excel sheets), might look nice to a lot of people, but I have been a programmer for nearly 30 years and in my experience, a good database is the cornerstone of all the rest … … and so far I have found nearly nothing on support for db’s in UE, Ok I can implement one via c++, but I don’t want to … … UE = BP for me. Like I said before, I don’t like the c++ framework in UE that much …
This approach gives me freedom, independence and of course a pure separation/abstraction between screenlogic and “other” logic. It also gives me the freedom if Epic go’s a way in the future I don’t like at all, I can switch to a concurrent platform and only need to modify/learn the UI part. Again, I have been around in computerland for many years …
I also looked at localization in UE and so far I’m not that happy with what I have found, so I’ll probably take that in Java/database as well. All this data throughput could end up being larger than the 5MB/frame at moments. I don’t know that yet, I will have to test that.
I my case, both (UE + Java code) will most likely run on the same computer and it will be used in a localhost-configuration as a sort of inter-process communication. Yes, slower (it is TCP), but therefore, I might need a larger capacity at moments.
For others, like Domotica creators, a similar approach can be taken. Make a 3D replica of the complete house in UE and via your BP’s, one can do the logic and steering in Java (or others of course).
I don’t think you fully understand what a beautiful gift you have created by offering all these powerful BP components at such affordable prices ! You have opened the way to allow all to integrate and communicate and that from within BP’s ! That was the main reason I was on your lips as from you released these !
I will use your 3 test steps and (latent) function calls.I will, however, not be able to help you with “testing on the other platforms”. I have a version for Android planned, but I will only start this earliest begin 2017. Sorry for that.
Cheers and thanks,
Geert
Ah, I never expected people to use the socket connections locally, the system was made with a client-server configuration in mind, although using it for local data communication should work just as well.
I’ve added a blueprint to the Socket Connection plugin to choose a custom buffer size, that will hopefully fix any limitations when using the plugin for local data communication.
Do note though that each latent blueprint (a blueprint with a stopwatch/clock in its top-right corner) will never complete in the same frame they are called, and so they will always takes at least 1 frame to complete. This is because UE4 only starts ticking latent blueprints at the start of the next frame.
Anyways, I’d love to see your project work, I think it would be very interesting, cool and especially funny to see all logic being done in Java on the same machine.
Fantastic! Thank you very much!
Thanks for adding the extra BP, I will see when the update will arrive !
I have not touched latent BP so far (used it only once or twice, without knowing all details), but will have a closer and detailed look at it begin next week ! Thanks for the tip !
If all goes according to plan (but it never does, does it ), I expect the first release end 2016, begin 2017.
For the hardcore c++ programmers reading this and thinking now “What a looser, c++ is 30 % faster than Java, so you are going the reverse direction …”. You are correct and I have done a fair share of c++ programming in my life, but in future steps of the project I will need AI on many different platforms (among them raspberry or banana pi’s for instance to do robotics steering).
I started making a framework in c++ to support this, but after months of compiling and recompiling for all these different platforms, there was always some version of a library that was not supported for some platform and that I however needed (like with boost for instance)! At the end I was using so many of my time in trying to support the different platforms (ask the people at Epic, they know what it is as well), that it was clear that I would have no time left to do what I had to do and that was completing my project !
After a lot of testing (again), I ended up with Java and after doing specific performance and durability testing in the range of my project (massive parallel), I was surprised in a positive way. If I had a lot of garbage collection going on, my overall performance drop in Java (compared to c++) was only 15-18 % and without massive GC it was even only 11-14 % slower. If you do the coding good, no more or less reliability problems in comparison with c++. The big plus for me about all this is that, out of the box, pretty much all platforms are supported and if I take pure jar-libs, they work on all these platforms, so literally, write once, use everywhere !
I do not intend to start a discussion on what programming language is the best as such a thing does not exist. For every job there is a range of programming tools/solutions (better and worse). I still love c++, but Java (and others) as well and if UE was written in Java for instance, it wouldn’t have been half as good as it is now !
Cheers,
Geert
OK, just had a look at the Java side and that is a massive, but nicely ordered, amount of code you have there, nice work !
The examples are well documented and easy to understand, thanks for that.
For newbies (to Java) it might be interesting to have one working example (Main() file), that would motivate them to do the extra mile afterwards (having seen it at work makes one eager to learn more 
).
You know I need to optimize code for my project and I see two possible bottlenecks/slow downers for my case. I’m confident that these would not be of any concern for more standard/average projects and I hope you do not feel offended by my rudeness. It is by no means criticism on your wonderful work !
First : I have not walked through all the code, but it looks like you made your own JSon implementation based on XML, correct me if I’m wrong. If I remember well, You mentioned before in one of the fora that “the JSon” you use is JSon-complaint (I don’t think it is fully RFC 4627 complaint, but that is not required). So, if I would use a lib like JSon.simple (extremely reliable and highly optimized parser, so super fast) to do the JSon part on java side, the outcome would be usable with the UE JSon BP’s ?
Second : Standard Serialization is a real bottleneck in Java (very slow). If I would use techniques that produce the same outcome of standard Serialization but with better performance, no problems raise on the UE BP side ?
Looking forward to the answers and thanks in advance !
Cheers,
Geert
The JSON serializer/parsing in Java is done with a library called jackson, which I included in the Low Entry Java library.
Which JSON serializer/parser you use doesn’t matter much for the compatibility between Java and UE4, as long as it’s valid JSON the UE4 plugin should be able to parse it.
The UE4 plugin uses a JSON serializer/parser made by Epic, which seems to be a custom solution made by them.
I don’t fully understand what you mean with the standard serialization part.
Java’s way of serializing things (like this: Java - Serialization) doesn’t work together with UE4.
If you want to send data between Java and UE4, you’ll have to use the methods (like integerToBytes(10)), or use the Low Entry ByteDataReader/ByteDataWriter classes (like .writeByteData().addInteger(10).getBytes()).
You can use different techniques to do this, as long as the data is formatted in the same way as the methods format the data.
I don’t think making your own way of serializing data will make a difference though, since the serialization methods already do as little as possible, there’s not much to change/improve there.
Also, if speed is a major issue, it’s probably best to not use JSON, but instead just send the data only.
Instead of this:
{“width”: 640, “height”: 480}
In code: create a HashMap<String, Integer>, put “width”=640 and “height”=480, serialize the HashMap to a JSON string, serialize the string to bytes
Send this:
640480
In code: serialize 640 with .integerToBytes(640), serialize 480 with .integerToBytes(480), merge the bytes together with .mergeBytes()
Or: create a ByteDataWriter with .writeByteData(), add integer 640 with addInteger(640), add integer 480 with addInteger(480), get the bytes with getBytes()
Then you can read it like this:
int width = .bytesToInteger(bytes, 0, 4);
int height = .bytesToInteger(bytes, 4, 4);
Or like this:
ByteDataReader reader = .readByteData(bytes);
int width = reader.getInteger();
int height = reader.getInteger();
The same can be done in UE4, with blueprints that have similar names to the Java functions.
Of course it is Jackson, I should have seen this, it is in all your dirs … … stupid me !
If you use “Serialization” in your classes (but I presume it will be mostly in the Jackson classes), it is an extreme slow way of serializing data in Java. That is all I meant with the remark on serialization. You can look this up if you are interested, diverse studies are made around this topic.
Thanks for pointing out that the JSon is actually not required ! Good old and fast bytearrays should do the trick even way faster ! I will have a look at what types of data (read/write) you do support and base on that ! I was somehow under the impression that I needed to follow the exact flow you had created to be compatible at both sides (java + UE). This should solve both problems I had at once !
Cheers,
Geert
OK, just watched your videos on all the plugins again and most is pretty clear, except for 1 thing :
In the File Manager plugin, you end by using this “Get Data (File)” node. Is the output a bytearray that can be converted to a string with your component, or is this a string output (text only files) ?
Apart from this, I just have one more question I have not found an answer for on the fora. Since I use a binary format to communicate via your TCP sockets (and possibly via your File Manager as well, if binary is supported ), I would like to add some structure in my “format” and one of the better ways, is to add a pointer table at the beginning referring to the beginpoint of every “object” in the bytearray. This pointer table will be an array of integers and I have not found a way to do a memory copy of that array to a bytearray (in one write/read you copy the complete array, extremely fast and efficient). Most programming languages have such a thing, but I did not find anything on it for BP online. Does such a thing exist to your knowledge in BP or will I have to do a (time consuming) for (each) loop to read/write the array (or write one myself in c++) ?
If you consider this beyond the scope of your plugins, I totally accept this, but one can ask !
Thanks in advance !
Cheers,
Geert
The “Get Data (File)” node returns a byte array (the filesystem plugin works with bytes).
I don’t understand what you mean with the second thing.
Do you want to send over an array of integers? Because the ByteDataReader/Writer can read and write integer arrays.
Anyway, the way I converted objects to and from bytes was with either one of these two methods:
Lets say each object has 2 integers and 1 string.
Method 1: Send an integer (object count), then for loop that many times, during each loop you send two integers and 1 string with the ByteDataWriter class.
Method 2: Send 3 arrays (1 array for each variable, so 2 integer arrays and 1 string array in this example).
Method 1 is faster (since you don’t have to create arrays first that are filled with the values of each object), but method 2 is more error proof.
If in method 2 not every array has the same length, you know the data is incorrect. With method 1 there is no way of knowing this.
I’d go for performance (method 1), since the data communication is local and thus errors are unlikely to happen.
I hope that helped.
The eclipse tutorial is done!
http://tutorials…com/
Anyway, to answer your other questions (that I somehow missed last time):
- When it comes to databases, many people use hibernate, I usually use my own database connection pooling code. It just depends on what you’re most experienced with / used to.
- If you have a unified way of handling data, then how do you know which data is what? How unified is it? Anyway, sending and loading data though Java (to save it in a database for example) will work, no question about it. It just depends on how the system is build that will determine how well it will work.
Many thanks!
A connection from a java program to MySQL is, apparently (I have it working) trivial. My intention is to have the java server receive the data from UE4, manipulate it as necessary, and then pass it on to MySQL.
The scheme that I have read about is to have the first byte identify the following data. For example, a first byte of 27 would tell the server that the following x bytes are formatted as an inventory item and this would result in a predefined processing of the data.
My thought is that I would also insert a predefined unique player identifier in the data sequence and flags for different operations.
As this server would only deal with player data, an example of my planned data structure would be PlayerIdentifier, Type of Data, Operation, Data in predefined format.
Of course, I’m a long way from a working system, but I’m confident that with the Socket Plugin and the examples in the source code (and much research) that I will eventually get it running.
Hello,
The LE Extended Library does not want to install on 4.9, It says its available. I click install via the launcher and it does nothing. It works for 4.11 and 4.10.
Ah that’s too bad…
I don’t think I’ll be able to fix that, since I’d have to send an email to Epic and make them fix it.
By the time they’ve read the email 4.13 will probably already be out (lol).
But seriously though, I don’t think it will be fixed, or at least not anytime soon.
I’ll send an email to Epic and I’ll post the response here, who knows.
Ah, this is what you mean with uniform 
My definition of uniform is that the server handles all data in the same way. What you described I’d call a protocol.
Anyway, I’d make a HashMap<Byte,YourDataProcessorClass> for this, with Byte being the type of the packet.
Then, when receiving a packet, you just retrieve the data processor for it and pass the packet to that.
This way you can easily separate code into multiple independent classes.
The data processor is just a class or interface with a function, each child class or instance has a different implementation for that function then.
This is basically how the server for my own game works right now.
I have to admit, this is entirely painless! As I am just now learning java, I expected a steep learning curve and a lot of trial and error, but this works out of the box!
I have a basic server (Example 1) running in Eclipse and have just copied the example Blueprint. Connection and Disconnection on cue and no problems whatsoever. Passing messages looks incredibly straightforward and I’m going to forge ahead into developing my database connections & etc.
This is just what I have been waiting for.
Thank you for making it available!
Is there a version that you can upload somewhere else for 4.9? I can compile one from github if you have that somewhere for download. Im stuck with 4.9 for the duration of my project unfortunately.