Multiplayer bugs becoming more prevalent - Harden & Improve Network / Multiplayer Support!

General Feedback & Requests
1

If folks have questions or comments, can they please put them in thread and not PM them to me! Thanks :slight_smile:

Although for most people moving to 4.10 was relatively simple in comparison to other releases, for me personally itā€™s been a bit of a nightmare. Iā€™m currently working on two Multiplayer titles that although ambitious arenā€™t doing anything outside realms of a normal competitive multiplayer game. However, a lot of time I find myself fighting issues purely because engine code hasnā€™t been fully tested in a Multiplayer environment, and having to come up with some pretty significant workarounds. One such example that has since been fixed was that you couldnā€™t change pawn you possessed in Multiplayer - something that is pretty commonplace in a lot of MP games.

So really Iā€™m just creating this thread to try and draw attention to some of biggest issues Iā€™ve been experiencing with Netcode both in 4.10 and below, in hope that Multiplayers getā€™s attention it deserves. Iā€™ve linked a couple of most major ones below but rather than just repeat myself, I really want to raise issue for need for a netcode hardening pass of some kind. As an example, hereā€™s a couple of issues Iā€™m hitting since 4.10:

Widgets Disappearing For Clients in Multiplayer
This is probably most frustrating of bunch and definitely surfaced in 4.10. Whenever a client changes Pawn they possess in a Multiplayer game, any sort of persistent User Widget stops ticking and therefore drawing too. I am yet to figure out why this is, but brought it to Nicks attention on Slack who indicated it may be related to Widgets being ā€œtorn down and rebuiltā€ when a player changes a pawn. I imagine thereā€™s a reason for this, but whatever it is, it doesnā€™t work at all in Multiplayer. My game has vehicles (pawns) that players can hop in and out of (via possession). All you need to do is add a widget to screen via a HUD class and test it in MP, and theyā€™ll stop working minute I try to posses another object.

workaround for this is god-awful. You have to force-tick widget by calling itā€™s Tick function from another class, which completely voids point of widgets not ticking when theyā€™re not supposed to be rendered. Once widget starts drawing again, itā€™s also ticking itself natively - so most of time itā€™s double-ticking. Needless to say, this is less than desirable.

Iā€™m trying to build a repro project for it, but itā€™s going to take a bit of setup because itā€™s Multiplayer. Nevertheless, answerhub post here: UMG Widgets stop ticking and/or drawing for Clients in Multiplayer - UI - Epic Developer Community Forums

Uint8 doesnā€™t trigger OnRep_ functions if value is Zero
This possibly occurs to multiple engine types and maybe even pre 4.10 - but Iā€™ve been using uint8ā€™s for serialization / quantization of values for replication but found that if value is zero, they wonā€™t trigger Replicated Functions for clients. This doesnā€™t just affect me though, since a lot of engine types do this natively (such as FRotators for example in FRepMovement) and therefore there will be some bugs in engine code. Again, to most people they wonā€™t be noticeable and there are workarounds, but this had me stumped literally for days and I feel like itā€™s an oversight in Engine netcode.

Answerhub Post here: Uint8 Doesn't Replicate / Trigger OnRep function if Zero - Multiplayer & Networking - Epic Developer Community Forums

ā€”

Itā€™s also worth mentioning this; There were a fair few changes to Replication in 4.10 and yet not a single one of them was documented. For someone working on two multiplayer-focused titles this has proven to been an absolute nightmare. New options for FRepMovement for example werenā€™t documented in release notes despite being a pretty significant (and useful!) change. only way to get around this was to compare relevant files on GitHub from version to version, which isnā€™t foolproof - or hope you run into them at some point anyway.

I realize most people arenā€™t doing Multiplayer and for majority of projects these kinds of bugs wonā€™t surface because I canā€™t imagine most people donā€™t have to be so conscious of bandwidth as I do - but itā€™s already taken a significant number of releases to fix some age-old netcode bugs (such as pawn possession for example), and it would be a shame if netcode didnā€™t receive same treatment as other areas of engine. I personally would like to see some updates that really focuses on Multiplayer in upcoming months. AFAIK, thereā€™s nothing planned on trello. Most importantly - Iā€™d appreciate it if changes that may affect multiplayer are tested in that environment as well as non-multiplayer ones. MP is after all a first-class feature of engine.

Many improvements to engine could be made, a huge stack of changes and optimizations that Pete has made for Unreal Tournament being brought into engine would be fantastic for example. ShooterGame and VehicleGame are fantastic resources that I can imagine took a lot of time and effort to put together. However, both are VERY old now (must be 2-3 years or more?) and neither scales very well past more than a handful of players. So on top of engine improvements, a newer up-to-date MP example would also be a welcome gesture :slight_smile:

@Community - feel free to post any weirdness you are experiencing in MP, and of course share thoughts on a MP hardening pass on engine.

List of Features / Fixes / Improvements requested thus far.

2

Thanks for posting this.

MP seems to be something of a red-headed stepchild in UE4. I, too, would love to see more effort put into multiplayer.

3

+1

ā€¦

4

I laughed :stuck_out_tongue:

Agreed!

My chat system disconnected sending client when message length >= 1024 characters (Fixed by implementing character limit): Client disconnects once replicated string reaches 1024 characters - Multiplayer & Networking - Epic Developer Community Forums

5

DateTime variables are not replicated at all :rolleyes:

6

I believe Iā€™ve experienced that as well, couldnā€™t get my day night cycle to work, had to make my own date time struct.

7

Iā€™m fairly certain thatā€™s intentional, since properties in that struct are not UPROPERTYā€™s and therefore not considered for Replication. By design you probably donā€™t want to replicate a struct like that anyway, it would be more efficient to replicate float/int that all components are derived from and have client build their own Date-Time from that on other side, otherwise youā€™re wasting a lot of bandwidth.

8

Hereā€™s a bug with rep-notify data when trying to replicate a struct which has values inside changed separately.

9

Hi,

thanks for this information! Can you enumerate some of bugs / issues that you found? I think itā€™s good to know for all user that want to start MP games.

Best regards,
Daniel

10

Hey Daniel/Polygon

I posted two major ones I found, but in terms of whatā€™s changed there were a couple of things Iā€™ve found so far. FRepMovement (Replicated Movement in BP) now has an option for what level of Quantization you want to apply to FVectors and FRotators, and by default they have far less precision than they used to. For most part you wonā€™t notice this, but I started noticing really ā€˜steppyā€™ rotation on my vehicles and particularly any object simulating physics and replicating, so I had to up precision to Shorts instead of Bytes. (uint16 instead of uint8). Itā€™s a nice feature, but surprised nobody put it into release notes.

Aside from that, most major issue Iā€™ve had so far has been with UMG widgets vanishing from screen.

There are usual glitches with Audio in Multi-PIE too, some of which have been present since beginning and/or pre-4.8, but I think audio engine rewrite thatā€™s going on right now should help with some of those.

11

Hey James,

thanks for quick reply! Iā€™ve not played around with quantized movement replication so far but Iā€™m a bit concerned that it might lead to a heavier stuttering behavior than before.
In 4.8 Iā€™ve done some tests with movement replication in a top down moba style game and old system caused a slight stuttering effect, too. So I hope that new options donā€™t make this issue even worse :frowning:

Another thing that Iā€™ve noticed in older releases was related to ping of sessions. ā€œGet Pingā€ node returns always 9999 for me, even if desired session that I joined ran smooth as hell - even via steam. Unfortunately I couldnā€™t manage to find some time to check this in 4.10 but I really hope that Epic puts some more effort in MP in general :slight_smile:

Best regards,
Daniel

12

Yeah there are a lot of problems with Ping node, which is bizarre seeing as it seems to work perfectly fine in C++. What you can try to do is get Ping from PlayerState, as that seems to be nearly accurate most of time.

stuttering youā€™re experiencing - are you using a relatively small scale for your objects? It may be that they donā€™t send updates often enough even, and therefore client receives a low percentage of packets. tbh, thereā€™s any number of things it can be!

13

Guys, why replicated expose on spawn variables not set on clients while construction script? How is that even works, that reason of construction script in this case at all.
Owner not set as well.

14

Thanks for those news.

Iā€™m stuck in 4.9 for moment, due to SSL Perforce issue but Iā€™m glad to learn those MP changes as it will highly impact our game. I must take care of those.

In MP chat did you succeed in getting all characters send via Network? By example, ā€œ|ā€ is not working and it truncated following characters. I didnā€™t had time to dig as Chat system in my game in low priority for moment, but if some of you have face this Iā€™m glad to know if itā€™s a bug to raise or not ^^

Speaking about Network MP, we also have stange log that I donā€™t know how to understand / Fix,
https://answers.unrealengine.com/questions/332523/how-to-fix-high-single-frame-packet-loss.html
https://answers.unrealengine.com/questions/332510/freplayout-history-overflow-forcing-history-dump-m.html

Also log on network that have been fix in 4.10:
https://answers.unrealengine.com/questions/329062/logsecuritywarning-invalid-data.html

15

They canā€™t be because it simply doesnā€™t work that way, Constructor runs instant object is created in memory, not necessarily when itā€™s in world - and itā€™s far too early to consider it for networking at that point. actor has to be in-play to have a valid net connection / actor channel. Additionally, as soon as you introduce even a few microseconds of latency - youā€™re entire solution will break. first thing to understand about MP programming is that it really is entirely different to programming a local-only game.

correct way to set this up would be to create a struct of all variables you feel you have to send immediately, and when client eventually receives them from server, they can run an update function which sets those vars accordingly and performs any tasks. You need to write your code in such a way that it can handle any networking situation, whether it be latency, packet loss, packet order etc. UE takes care of latter two for you mostly. Owner also has to be set from Server, and if client has no knowledge about that actor yet, it canā€™t set that up.

EDIT: You can also use Deferred Actor Spawning in C++, which will allow you to setup properties before actor sends itā€™s initial replication packet.

Iā€™ll be sure to post here if I find any issues, I havenā€™t got as far as implementing a chat system yet! Are you using FString to replicate data? FString shouldnā€™t suffer any truncation AFAIK, but FNames will since they are case-insensitive (but cost have bandwidth of FString).

EDIT:
As for History Dump overflow - I believe thatā€™s caused when vars or RPCā€™s that are queued for replication start to overflow buffer, and therefore you have potential for clients to lose sync. i think thatā€™s what it is anyway but I havenā€™t found an easy way to debug it. It would be quite nice actually, if network profiler showed this kind of stuff as well. Itā€™s super-rad for monitoring bandwidth but a little extra profiling wouldnā€™t go amiss.

MP, especially in PIE is just a pure nightmare to debug if youā€™re in Visual Studio. Itā€™s actually easier in BP because you can select debug object and debug world (Client 1, Server etc.) Not easy to do in VS :frowning:

16

I dont understand logic behind this. Server, one who spawn actor, server know expose on spawn variables in construction script. Then server call client and Client one who create replicated copy of actor, but does not have access/ dosnt know expose on spawn data?

ā€œCorrectā€ way doesnā€™t work with projectile movement component initial speed.

17

This reminds me of a feature request I made: Networking pass-through nodes. - Feedback for Unreal Engine team - Unreal Engine Forums

What do you guys thing about something like that?

18

I commented on your link but I shall here also just incase. What you want is Switch Has Authority node :slight_smile:

19

You can Spawn objects and set properties on them straight away, but they wonā€™t be set in construction script because construction script runs before all of that happens. Think of construction script as a ā€œI want to create one of theseā€ - thatā€™s what it is really, almost like ā€œhereā€™s a template - go make oneā€. This is more apparent if youā€™re familiar with C++ or something but it is by design. Blueprint kind of hides that sort of thing away from you a bit.

IDK, Iā€™m not very good at explaining it :stuck_out_tongue:

As always it depends on situation but, in case of a projectile, initial speed is always same right? Even if not it doesnā€™t matter, since if projectile is spawned on server and has itā€™s velocity set in that same frame, client will also have that same velocity as soon as object is created, so long as youā€™re replicating movement of course.

20

nope, init speed is different, i calculate in on spawn to hit target here vid
And i expose target location on spawn, turns out i missed to turn in replicate movement, so after init spawn server would fix movement, but idea is, i cannot use expose on spawn variables over network on client.

also if you know better way to fire projectile, tell me please