TLDR: Animation desync and positional desync due to using locally spawned actors on the client and server in the same position. Help?
Design:
Multiplayer game where bases (walls, foundation, etc) can be placed along with random terrain generation (think the game Rust). The bases (walls etc) SHOULD BE a HISM/ISM (Instance Static Mesh) so they can be batched on draw calls.
I don’t want the bases to be deleted when they go out of range.
I don’t want to send a huge amount of data to players when they connect about the terrain…they should just need to use a seed and noise.
So neither are replicated.
The Problem:
This all works perfectly locally/single player - but multiplayer… the player goes on the terrain and the limitations of the unreal engine start to show themselves. The animation desyncs begin as well as once I was able to get the client proxy (not locally controlled client) to be very far away from where it should be on the server and on its local client.
Some more info:
I understand I’m doing something unreal doesn’t want me to. But I have to be honest my design approach should be fine. I understand that I may have made a mistake with floating point precision and a slight mismatch between terrain generation. But I don’t think this is the case as when I walk on top of the flat foundations of a base I can get stuck in the falling animation.
The Solution
Try another version of unreal engine? try Character Mover 2.0? Try hard setting the ground I am standing on hacking away at the character movement component? Ditch the Character Movement replication and brute force send all the data myself?
The Real Solution
quit.. I mean in all seriousness I am at my limits with unreal. It does save so much time over Unity having a lot of things already solved. But the way these are solved are incredibly rigid and offer limited scope. I have 10+ years using Unity and have been learning unreal engine for the last 4 years. I was so happy when I had several of my systems working and now am completely heart broken to so see the animation and movement desync. What are my options manually control movement and animation using RPC and On_Reps and attempt to brute force animation and positional syncing? I really wish Unreal would just be more flexible… I just dunno.
More info on what you mean by animation and movement desync is needed.
There are 2 broad definitions of desync between gamers and devs. The most common is simply lag which gamers believe the server is out of sync.
Player 1 applies movement, CMC executes it locally for responsiveness (client-side prediction). Moves & Player 1’s results are sent to the server. Server applies movement and compares results. Location/Rotation within margin of error it sends an ack. Otherwise a correction. All others receive the servers result.
Now think about each players connection. Ping, loss and jitter. It takes time to get the moves to the server, processed and then received by other players. each of those other will be receiving the updates at different times. A player with a 50ms ping will get the update in 25ms after send. A player with a 250ms ping will get it in 125ms.
Each players simulation of the world is always a random point in the past compared to other players and the server.
The other definition of desync is when the server itself is starting to choke. Servers have a hard tick time. Typically this is 30Hz, so an interval of 33.333ms per tick. If server load increases enough a tick can start to take 40, 50, 60 etc ms.
One off events that surge the tick for a few cycles and then relax don’t cause serious issues. Counter to that if the server load is persistently high the tick interval can cascade and eventually cause the server to crash.
Using locally spawned objects should be fine unless they can affect movement. The problem is if you apply a wall and the packet to telling the server to add a wall is dropped, then movement on the server will not be affected.
Remember the client server model. Server is the movement authority.
Player wants to build something. client RPC’s the server to build. Server uses the current position (look at), does the needed traces and spawns a “replicated” actor at the given location. The Actor being “replicated” will replicate down to all clients.
Everyone has the same exact location for the actors and their collisions.
To overcome the “they disappear at n distance” issue, you have to increase their network cull distance. By default it’s 150m.
@Rev0verDrive thanks for taking the time to respond.
To answer your question about the desyncs:
The clients tells the server to spawn an object. The object is spawned on the server and the client then receives a message that it can spawn its own version (I do this so that data is NOT sent multiple times when nothing has changed, due to unreal limited design they want to delete and resend data constantly… Don’t know why… and yes all the data/objects are fully deleted when you go out of range… There’s not even an option like “hide” so that I can say only replicate what is changed. And no I cant have it always relevant otherwise a million bases and object will be sent to connecting clients bricking the server… Anyway.) So there are two version of an object that are not replicated. One on the server and one on the client both are identical in theory (same position rotation etc). On the server (listen server in this example) this player sees everything correct. There are never any problems with movement or animations because it is the server. I fundamentally and fully understand why it is authoritative this is awesome.
On the client we have the other player (the listen servers client) move around and jump. Occasionally while on top of the the object that is locally spawned we see the player jump and then when they should land and return to the walk/run animation they stay in the landing animation and move around. When looking at what the server see it sees everything correctly. Which is odd since I thought it was supposed to be authoritative. I would expect these results if that was what was happening on the server as well but that is not the case.
I was under the impression that client proxies (clients not locally controlled) do not run their own true movement calculations and are just sent them over the network. This however is likely untrue seeing as we see a mismatch between the client and the server. The movementmode is not correctly replicated.
Previous Findings:
On an interesting point while building this system a different way (I have built this system 5 different ways over the years following my intuition that not constantly replicating the same data over and over would lead to better results…anyway) I was able to get objects that were set to be bNetStartUp objects and had some interesting results. I never continued this approach but want to report my findings here. I was able to get the NetGUID to sometimes be wrongly assigned to its objects. The NetGUIDs were confused by the order they were spawned so some object could have there NetGUIDs reversed which would cause when you walk on these for you to be instantly teleported to the other object were that NetGUID was assigned. Which shows that multiplayer/replication expects for the ground to be consistent and known.
Why am I doing this:
I think it seems pretty intuitive that sending data over and over (going outside netculldistance) again about how a base exists when nothing has changed is poor design. I think it seems obvious that sending huge amount of data about how a world exists is poor design and that an approach with noise and a seed is optimal. I want to achieve these because having some things be close enough to what the server sees is important for what else I want to achieve in the future. Specifically small armies. I want to give the clients just a single location for where the army exists and locally control what the client sees. Similar to how say you crash a car in GTA each client does there own damage and it looks different each time. This is the approach I need for what I really want to achieve.
What I am trying to achieve:
That a player in a multiplayer setting can navigate over objects that are locally spawned. Unfortunately If I want to proceed with this I will need to basically remove everything that exists in unreal. That being its replication system and movement that is done by default. Which is awful since I can see so many things and the chain reactions caused. I believe I know exactly what the issues is but am unsure how to solve it. The floor/ground the player is standing on should be the key to solving this. If it is trying to work with a nullptr or ground that doesn’t exists it could explain the problem. The only thing that is really annoying about this is that the player doesn’t always get stuck in the falling position.
I know what unreal wants me to do:
I completely understand what the basic/default design pattern is for the base system. Just tell the server to spawn a replicated object… Okay. But I feel if I was a better more knowledge hacky unreal dev I could get this to work locally. I also don’t understand how anyone could even build a game like minecraft or rust in unreal. How would you make it multiplayer. Surely not sending these massive geometry and objects around via the network. That just seems crazy to me. I can’t honestly believe no genius has solved this already.
The future:
I really really don’t want to quit what I have built so far. Its honestly amazing besides the animation and movement desync. I feel what I want to achieve is something 1000’s of unreal devs want and is not currently easily available. I have no doubt a AAA team in unreal could make this work and have minimal impact on the server/network because of it.
If you are worried about the amount of data you need to replicate, Dormancy might help:
I believe Fortnite makes extensive use of dormancy. Once an actor has been replicated, if you tag it as dormant, the server will skip it when checking for replication updates. This is especially useful for static elements such as your base building blocks.
UE5 also introduced a push-based replication system called Iris. This is still an experimental plugin, so use at your own risks, but the idea is that instead of constantly checking all properties for replication, you as a developer are responsible for tagging a property dirty when it changes. Only then will Unreal replicate it. This can potentially allow you to be much more loose about cull distance and the amount of actors to replicate. This is C++ only, though.
My theory is that your system causes issue with the replication of character’s ReplicatedBasedMovement variable, which contains a reference to the component character is based on. Since these actors/components are dynamically spawned but not replicated, they cannot be referenced over the wire, so there must be some nullptr issue indeed, causing the falling state. It may not always happen since the simulated proxy probably also tries to predict landing/base, so there’d be a race condition between prediction and replication…
This is a long shot, but can you enable Verbose logging on CharacterMovement and see if you get this message ?
UE_LOG(LogCharacterMovement, Verbose, TEXT("Base for simulated character '%s' is not resolved on client, skipping SimulateMovement"), *CharacterOwner->GetName());
Also, try overriding functions or binding delegates in your character class and log some info to figure out what’s happening
UPROPERTY(BlueprintAssignable, Category=Character)
FLandedSignature LandedDelegate; // bind this
UPROPERTY(BlueprintAssignable, Category=Character)
FMovementModeChangedSignature MovementModeChangedDelegate; // bind this
virtual void OnRep_ReplicatedBasedMovement(); // override this (don't forget to call Super)
UE4 is using names to match stuff when replicating. I had issue in the past with a plugin that was generating static meshes, in editor time, but it still wasn’t net addressable, so when a player stood on that static mesh, it had the same issue.
For that, I set bNetAddressable on the component, which will make the replication driver believe that it can be addressed over network. You just have to make sure that it’s name is the same on both sides.
That was for component, but I see that there is a SetNetAddressable function on actor too, so you can try that. Just make sure they are spawned with the same name, both on server and on clients (maybe use a name combined from XYZ coordinates?), then call SetNetAddressable.
I was aware of SetNetAddressable() but on review my previous code (older version) I noticed how I was implementing SetNetAddressable and was calling it in the constructor which is to late in some conditions for the object to be complete stably named. It needs to be done using a deferred spawning. Honya you also gave me confidence that SetNetAddressable was the solution so I applied it every actor and component and IT WORKED!!!
You literally saved my life Honya. Thank you so so so much <3