Low Entry Plugins

This thread will make it easier to talk about the Low Entry plugins.

Feel free to post whatever you want. That includes questions, requests, etc.

Hi there,

First of all, great work and thanks for sharing this for free !

The main question I have right now is how you connect via your BP’s to the java server (are there “connection BP’s” in your BP’s or similar) or do I have to set up my own in c++ or BP for instance ?

My hope is that I can get (almost) anything in UE done via BP, so also the connecting to the server and receiving and sending as well. That way, I only need one language in Unreal.

I have seen a number of “BP for TCP/UDP” related topics/tutorials in google searches, but no one is actually available (at least not to my knowledge).

Thanks in advance for clearing this out !

Cheers,

Geert

Hi Geert,

There will be more Low Entry plugins (6 more to be exact) available on the UE4 marketplace soon (1-2 months from now Epic told me).
One of those plugins is the Socket Connection plugin, which allows you to easily connect to and communicate with a server (that uses the same protocol as the UE4 plugin and Java library do).

So, you’ll have to wait and see till then!

Also, if you want to do everything in blueprint you’re in luck, since every Low Entry plugin is quite heavily focused on blueprint.
Using the Low Entry plugins with C++ should be possible as well for the most part, but making everything possible in blueprint is the primary goal.

Will the socket plug-in only be able to connect to a Java server?

The socket plugin uses a certain protocol to make communication with a server far easier to do with blueprint.

The socket plugin can connect to any server, as long as it uses that specific protocol.
If you program the protocol in Python for example, the socket plugin will be able to connect and interact with the Python server.

The protocol is fairly simple, so it shouldn’t be too hard to implement.
The protocol is described in Java as well as in the UE4 blueprint documentation (the mouse-over text).

The same counts for the way data is converted to bytes and back.
If you would like to use byte data that was created using the Extended Standard Library’s byte data/writer blueprints on a server, you’ll have to implement the same encoding logic on the server side as well.

The same counts for encryption, compression, etc.

If you’d like to use a different language for your server, it’s definitely possible, you’ll just have to implement the same protocols and logic in that language then.
That might sound like a lot of work to do, but the majority can just be copy pasted over from the Java library source code, depending on what language you use exactly.

You said earlier in the thread that this plugin would be released by Epic in 1-2 months. Might it be possible to acquire (as in download from Dropbox or similar) a pre-release copy to start learning how it works?

I would like to, but sadly this isn’t possible no.
Luckily the Socket Connection plugin is very easy and intuitive to use and so it won’t require much learning.

Screenshots of all the Low Entry plugins can be found here: http://public…com/files%2FLowEntryUE4%2Fscreenshots%2Funzipped
If you scroll down, you’ll find the SocketConnection plugin screenshots there.
That will hopefully give you an idea of how it works.

Hi ,

Just had a look at the screenshots and you rock ! You made my (and I think a lot of guy’s) life a lot easier ! Looks good and easy to use ! I will be happy to wait for another 1-2 months to use/test these as I need till September to do the main Unreal part for this project.

To be honest, I couldn’t have imagined using JSon in BP’s :slight_smile:

Keep up the good work and I’m looking forward in using these !

Cheers,

Geert

Just bought the first two available (JSon and Compression) and looks good ! Hope the others arrive soon so the real testing can begin !

I noticed that all video’s posted on your Youtube channel are 9 months old. Can I ask why it took this long to be available in the marketplace ? Just out of curiosity :slight_smile:

Cheers,

Geert

The plugins were made a long time ago (in 4.2 or 4.3 I think).

Initially I made them to support my own games, since the games that I wanted to make weren’t possible with just blueprint alone, and I really didn’t want to be bothered with C++ because it’s slow to compile, it’s slow to get the intellisense info, it has a very high risk of memory problems, etc.
Later I noticed that the plugins were actually quite useful for about any project really.
Then I thought that perhaps I could sell them, to get some of the time/money back that I had to put into it.

The plugins were somewhat done already about 11-12 months ago (they were still in a very rough/beta-ish state back then though, the plugins have matured a lot since then).

Anyway, the thing is though, Epic’s code plugin marketplace system wasn’t done yet back then, so it was impossible to have code plugins on the marketplace at that time.
Code plugins on the marketplace just recently became possible, which is why the Low Entry plugins just recently became available on the marketplace.

Glad to hear that, this should mean that they are already mature in nature and less errors are to be expected !

I know what you mean with the c++ vs bleuprint dilemma, I had the same thing. I’m a c++ programmer (and Java of course as well) and I love c++, but the framework in Unreal is just not my cup of thee. I’m grateful to Unreal they provide us with UE ! It would take me years of programming to get at that point and yes, I will admit time after time that I’m in love with the graphics produced by UE (I know how to program program OpenGL, but I could never reach such results) !

On the other hand, sometimes I do notice the c++ limitations of the framework and one can chose to start adding own external libraries (eg db access or for instance ZMQ), but that is where things get complicated in reality. Not only do you need to support it for yourself on diverse platforms (eg Windows, Android and iOS for starters), but changes made by Epic between UE-versions dare to be catastrophic as well, like with the SQLite support for instance. One version it is supported (out of the box - not officially although) and newer versions it no longer is (at least not to my knowledge).

Personally, I do not get why, for instance, Boost is not supported out of the box, smart-pointers and a lot of other very useful stuff becomes immediately available to the programmer at that moment for pretty much every platform out there and I think a large part of the c++ community knows how to work with Boost …

But let’s not make this an A vs B topic and I’m glad Epic provides us with such a wonderful tool, capable of both c++ and BP, and people like yourself provide us with the tools in BP to go that extra mile further (in BP) without having to go into the UE c++ framework !

And to be brutally honest, for these prices on the marketplace, one would be stupid not to buy them !

So thanks again and looking forward to discover the other plugins !

Cheers,

Geert

Any word on when the socket plugin will be available?

I have no idea when the Socket Connection plugin will be available, I didn’t even know the Compression and Json plugins were live until I had read this thread.

I expect the other plugins to be available on the marketplace soon, which is a lot sooner than I expected :smiley:

As of today, the socket plugin is available as well (and encryption too) ! Look in the new content for now, it will take some days to be in the “normal” content.

I have bought all LE plugins now and next week I will be testing them out in depth (preserved a complete week to play with them and find out where the limits are in both UE and Java).

Just a few questions :

  1. If I remember well, in one of the answers you provided on one of the topics, you mentioned that the plugins are 4.12 ready, but even with the new ones posted today (like the Socket Connection Plugin and Compression plugin), still no 4.12 version. When can we expect the 4.12 version ?

  2. In your Socket connection plugin, is there an internal buffer for new messages that are not handled yet and if so, is there an overflow mechanism like for instance in ZMQ or do we have to handle all incoming connections our self’s ? And, in that case, what happens when an overflow occurs ?

Thanks for the components and looking forward to test them out !

Cheers,

Geert

Yes, when will the 4.12 versions be available?

Will the Java source code be available?

I have no idea when you can expect the 4.12 versions to be available on the marketplace. It’s not in my control. All I can do is send the updated code to Epic and hope they’ll put it live soon, the rest is Epic’s responsibility.

The Socket Connection plugin works like this when it comes to buffers: there is a 5 MB (megabyte) receive buffer, and an unlimited send buffer.
This means that you can maximally receive 5 MB per frame (because during Tick the buffer is emptied), this means that with 10 FPS you can maximally receive 50 MB/sec, 60 FPS gives you a max of 300 MB/sec.
This buffer can’t overflow, because TCP (the protocol it uses) sends packets to the sender about how much space it still has in the receiving buffer. When the receiving buffer is running out of space, the sender will halt sending data until more space becomes available.

Also, know that with ‘buffer’ I mean the internal buffer, in other words, the cached bytes per frame. During each frame the received data will be placed in an UE4 bytes array where it will remain until the full packet is sent, then this bytes array will be given to you in blueprint.
So having a receive buffer of 5 MB doesn’t mean you are limited to 5 MB packets. You can for example send 50 MB packets as well if you would want to, there is no maximum to the size of the packets you can possibly send. The 5 MB only refers to the maximum amount of data per frame that can be received, not per packet or anything else.

Anyway, the same pretty much counts for Java. Each client has a receive buffer, which can’t overflow. When listen() is being called on the server, it will go through all client buffers and read the data out of it.

The Java source code is already available.
The JAR (which contains the source as well) can be found here: http://public…com/#files/LowEntryUE4/java
The code repository can be found here: http://internal…com:5000/Applications/Plugins/Java/LowEntryUE4

Knowing Unreal, if a major release is released (like now with the 4.12), it can take several weeks up to a month before all packages in the marketplace are updated! I had a similar experience with something I bought on the marketplace just when 4.11 came public and it took a month (the package owner was nearly desperate as a lot of his customers couldn’t laugh with that, but he couldn’t help it at all !) … … every major release is of course a serious load on resources for Unreal …

Good to know it is TCP. So, I will have to do the load balancing on the java side. As I will use an actor system like akka in Java and can end up with many simultaneous requests. I will have to make a Que at the Java side.

I presume the 5MB limit is defined by the code you are using from Epic and no modifications can be made to it? In that case, if I need to go over the 5 MB/frame limit, I can open diverse sockets (on same IP for instance) but on different ports and handle this like with UDP packages (self handle the re-assembling and order of the received packages) ? Is that correct ? And just for all clarity, is there a (coded) limit on the number of sockets in UE that can run simultaneous ? I have done socket programming before and know that it is easy to stall a program when using too many of them, but just to be informed.

Another question : How are time-outs, errors and broken connections handled on UE side ?

Thanks for the background info, that will help to advance faster !

Cheers,

Geert

The 5 MB limit that is set on the buffer is set by me, since I figured, even at just 10 FPS the client is capable of a maximum of 50 MB/sec. Most people’s internet won’t even reach 5 MB/sec, let alone 50 MB/sec. Having a higher buffer has the downside that it uses more memory, so systems with less memory, or many socket connections open at once, can cause problems with that.

If you really need a higher speed than 5 MB per frame, then you can indeed have multiple socket connections open at once to the same server and same port, but I don’t see a possibility where that would ever be needed.

If the 5 MB/frame ever becomes a problem I’ll add a parameter to the Create Socket Connection blueprint to allow you to choose a receiving buffer size yourself, but currently that would over complicate something that you shouldn’t be bothered with (it gives you a reason, or even the need, to research what a receive buffer is, what the implications of a low or high buffer size is, etc, things you shouldn’t be focusing on or spending time on).

Every tick the Socket Connection plugin checks each Socket Connection to see if it still is connected, and if it isn’t, it will call the disconnected event.

The thing is though, is that TCP timeouts and disconnect packets are currently handled a bit weirdly in the UE4 socket code (made by Epic).
Instead of correctly handling TCP timeouts and disconnect packets, they just ignore them, and then the connection status still returns ‘Connected’, even though it isn’t.

Hopefully they will change this in the future, although I haven’t tested it in anything after 4.11.0 yet, so maybe it’s already fixed.

Either way, the best way to deal with timeouts and disconnects is to use Function Calls or Latent Function Calls, since they check for timeouts as well, which will always work no matter what.
If you send a Message, send a Connection Validation as well (or set Automatic Connection Validations to true/on, then it will send one every 30 seconds automatically).

Lastly, know that the socket code can work slightly different on every platform, because the low level socket connection implementation is different for each platform.
The biggest differences can be found between HTML and the rest, because HTML only supports async sockets, whereas the other implementations rely on being synchronizable. The other platforms should be pretty similar to each other.

Idealistically speaking, you wouldn’t notice a difference at all between different platforms, which is how it should eventually be, but currently there are some differences still.
For example, some platforms might handle disconnects properly while others ignore them, and some platforms might fail immediately when trying to connect to an invalid URL/port whereas some platforms return true (‘Connected’) even though they didn’t actually connect.

Sending and receiving packets will always work the same way on every platform however, the differences can only be found in the lower level workings of the socket connections.

Anyway, to test the major low level differences, you can do this:

  1. Test disconnect packets: on the server side call disconnect() onto a client, then see if the UE4 client fires the disconnected event.
  2. Test timeouts: while being connected, shutdown the server (kill the task), then see if the UE4 client fires the disconnected event.
  3. Test connecting to an invalid server: try to call connect on an invalid IP and port, like “example.com” and 5555 for example, then see if the connect blueprint returns false.

Some platforms will fail these tests, but know it’s probably best to not build workarounds for this yourself, since these differences will eventually be fixed and ironed out.

If you use Function Calls, Latent Function Calls, or Messages + Connection Validations, it will catch a fail of any of the tests described above.

Finally, if anyone can run these 3 tests on different platforms, please do and post the results here, that way I can see if there’s something I can do about it.
I’ve only tested Windows and HTML thoroughly (and only on 4.11.0 and older), so if anyone could test this out as well on different UE versions and on different platforms it would help a lot.

Just picked up the Socket plugin.

I’ve gotten the source code for the Java server and am starting to attempt to use it in a Netbeans project (which I am also just learning to use.)

Just to make sure, the source code download from your link is all inclusive? Does it depend on other LE code?

I ask because that’s the first question I ask when I try to determine what I am doing wrong. :slight_smile:

The source code of the Java library requires no external libraries, everything it needs is in there.

You do have to be certain that both the src and libs_src folders are used as source folders.

I don’t know how you would do that in Netbeans, I use Eclipse myself, although that shouldn’t matter that much.