Suggestions for a refactor of Character Movement's Netcode, for better diversity.

So recently I wrote my own version of a networked movement component, based heavily on the CMC for my vehicles. See here.

it’s neither finished nor perfect, but something I noticed while going through CMC and trying to learn how it worked, was how convoluted the netcode is for Character Movement. Netcode is split between the Player Controller, Pawn and Movement Component itself - and trying to disect how it works was extremely difficult. It took me a solid couple of weeks of programming on that task alone non-stop, and slow study of the source code (while writing out my own component based on similar methods) to create a more diverse ‘networked movement component’ for my vehicles.

The result was worth it though - I now have a base movement component for any kind of vehicle that handles prediction and reconciliation, and can support different input. By overriding it and and a few functions within, I’ve got predicted movement for flying, hovering, tracked and walking vehicles with relative ease and minimal duplication of code, and I feel as though some of this can be applied to CMC to separate it into a more diverse and reusable ‘networked movement component’, which with little work from another developer could be extended to work with almost any custom movement. My code is by no means perfect, and I still want to split it up more - but it feels like a step in the right direction and Epic’s programmers could no doubt do a better job than me.

I don’t really have many specifics but I do feel like this could be an enormous benefit to the engine, so here are some ideas based on what I can remember from doing it:

  • A LOT of the ‘Prediction Data’ can be reused for multiple objects, without the need for each one to process and verify it’s own timestamps and moves etc. A suitable place for this would be the GameNetworkManager, which could cover many of the tasks that CMC does, like ‘speedhack’ protection, timestamp verification, storing and updating ‘FNetworkPredictionData’ for both client and server etc. This would dramatically reduce the complexity of CMC and also enable the developer to more easily predict data for other objects, such as weapon projectiles etc. UT seems to have extensively modified CMC so it can store extra information like whether a projectile was fired that frame etc, which seems like a roundabout way to solve a problem and is again very one-dimensional.
  • CMC ‘Assumes’ that it’s owner is a Character, with a capsule root component and a skeletal mesh. This is very limiting if you want to try to modify it for anything other than a Character of similar size and shape. I’ve managed to create mine so that it just requires a separate pair of primitives, one for collision and one for visual representation (required for smoothing updates). My vehicles use a ‘Skeletal Mesh’ as their root object, and it would be nice to change the smoothing code to be able to use the Physics Asset as the collision primitive and offset the visual mesh from that collision for smoothing (which is what I’m looking into). This would cut the amount of scene components required in half and alow for more complex shapes. Once again this would make it easier to diversify the component for any size and shape of object.
  • CMC mixes reusable netcode and movement code, which could be easily split into a base ‘networked movement’ class and a child ‘character movmenet’ class. Functions like ‘PhysWalking’ and ‘PhysFlying’ for example would be part of the Character Movement class, while ‘Server_SimulateMove()’ could be part of the base class. The current ‘PhysCustom’ method is another roundabout way of solving the diversification problem. The fact that UT has had to modify the engine that was designed AROUND it should be a testament to how one-dimensional and non-extensible CMC currently is.
  • Code is separated between the Player Controller, Pawn and Component. Lines 3879 > 3903 in PlayerController for example could easily be moved into the GameNetworkManager, which feels like a more suitable place for updating Server Prediction Data. Again it feels like the PC is very one-dimensional, assuming the players’ pawn is a humanoid.
  • This is a personal gripe of mine - all netcode and AI code in Movement Components assume that “Rotation Follows Velocity”. This rule applies fine for any two-legged biped, but isn’t sufficient for almost anything else, even the most basic vehicles. Even the basic pawn class has a lot of ‘bloat’ because of this, with variables like the ‘InputVector’ and functions like ‘ConsumeInputVector()’, both of which assume the only input required is a world-space directional vector. ‘RequestDirectMove’ (called via UNavMovementComponent for the AI pathfinding), also assumes that all you need is a world-space ‘velocity’ to the next pathpoint - which is quite frustrating and limiting when creating your own movement. Passing an overrideable ‘struct’ instead of a vector to a lot of these functions would make them much more diverse.

In my case, I send an ‘FRepVehicleInput’ struct to the server for predicted movement, instead of a world-space quantized vector. That way if I have a vehicle that needs extra input data, I can just extend this struct and use the same functionality, instead of duplicating it.

  • Finally, a lot of CMC can be separated into different files - the same way AActor is. I for example have ‘BZGame_VehicleMovement’ which is a 2,000-line class with all server/client functions and basic functionality for applying velocity and forces to an object. I then have ‘BZGame_VehicleMovementReplication’ which currently stores all struct functions, like ‘FSavedMove_vehicle’ or ‘FNetworkPredictionData_Client_Vehicle’ etc. Again some of this can be split out in to the GameNetworkManager as stated in point 1, which would allow multiple classes to reuse the same code.
  • Bonus points are available for creating a physics-based movement component too, which easily supports move replaying etc. I had to give in when doing networked physics, and instead ‘simulate’ physics in my own code. PhysX does all this fancy SIMD stuff that makes it’s collision and solver code much faster than mine could ever be - and it would be great to benefit from that. My idea would be to run two ticks on the component, one pre-physics which calculates and applies all acceleration & forces to the existing physics velocity / angular velocity - then a second post-physics tick which sends the resulting data to the server / out to clients, and where clients compare recieved data from the Server. CMC currently also calculates it’s own collision rejection, and it’s really not that great (or fast) - it can only resolve for two walls maximum which doesn’t suit my vehicles and pays no respect to physical material properties, like restitution or friction etc. As someone making a vehicle based FPS / RTS hybrid - this is of critical importance to me, and the speed difference is required due to the potential for huge numbers of units (way more than Paragon can currently handle, which is discouraging!)

Not expecting this to happen overnight or even in the near future, but I feel like this would be a huge benefit to the engine and pull it away from the Stigma that it’s been designed entirely around making Shooter-Games. UE4 has such a strong heritage in FPS games, but I’d like to see it’s AI and Netcode diversified beyond playing as a two-legged human!

Food for thought :slight_smile:

I fully support what TheJamsh has written, if there was one area that UE4 could benefit from improving more than any other - IMO it’s here. Only the most basic of games don’t require client-side prediction or can work with CMC in it’s current state and it’s incredibly limiting.

As for CMC in general, it’s a behemoth and a stone-wall to most people who require prediction with out-of-the-ordinary gameplay. Even studying how it does things for your own purposes is a daunting task, it really is Alice’s rabbit hole.

I’d definitely want CMC to be pulled apart and made into several components as well. Like TheJamsh, I’ve had to make my own custom movement component for physics based movement as CMC doesn’t support this (no torque simulation and it’s really painful to modify without completely breaking it). Additionally regular physx rigidbodies don’t allow proper netcode for fast based multiplayer games which basically doesn’t leave you with any other options than to a) make your own thing b) mod CMC c) mod engine code to get physics to replicate and predict properly.

CMC is a mammoth on Unreals architecture. CMC’s c++ class file alone is over 10 000 lines of code which (in my opinion) shouldn’t be something we see on world class engine like UE4 when that could still be handled properly on smaller more reusable components.

Thanks very much for the great write-up, I think you have some very good ideas. I’ll certainly discuss this with the team, and see if we can tackle some of it next year. My biggest concern initially is how well we can maintain backwards compatibility with some of these changes.

Hi James,

That’s great news! I think it would be preferable to leave CMC where it is and start fresh. Gives you a lot more room to work with and solves any backwards compatibility problems.

Awesome, thanks for the response!

I too was thinking about that this morning, since a lot of these changes would ultimately break existing projects using the current CMC which is of course a major engine component / feature in itself. My only solution right now would be to create these as completely new classes, and maybe phase out the old ones over time so that people have the time to ‘upgrade’ or port their existing projects.

Perhaps if there was a ‘Predicted Physics Movement’ component that works as described in my last suggestion, the character class could eventually be updated to use that component instead of the existing CMC - ultimately treating the character class as a PhysX actor. Applying velocity / rotation on the object as if it were a regular PhysX actor would bring quite a few advantages, like interacting correctly with physical material properties, collision rejection from >2 surfaces etc without getting stuck. It would also interact nicely with the physics nodes like ‘Add Force’ and ‘Add Torque’ etc, which CMC currently has it’s own versions of.

Making that component then support any primitive object for collision and mesh (or using the physics asset from a skeletal mesh and offsetting the visual mesh for smoothing) would certainly help with supporting non-character pawns.

When thinking about the movement component use for something other than characters, we could really use a fixed timestepped physics on it.

I’ve done this myself for my own custom approach, mainly because things can move really fast with vehicle physics. Even a small difference in physics calculations would require quite big corrections from servers side. To be able to simulate fast paced physics on both client and server, you really don’t want the framerate to play any role on how the physics get calculated. As far as I know, CMC has substepping option, but it still splits the time in variable steps rather than run them all at same fixed timesteps.

If CMC would ever get redesigned, I’d really hope to at least have an option to run the it’s physics at fixed timesteps (also wish Unreals PhysX integration allowed this as well but that’s a whole another topic).

Its a tricky issue but the sooner you get major changes like this in there the sooner everyone can be on the same page, your suggestion is a fix and shouldnt be treated like a regression! I know it creates extra work but Epic had a chance earlier in UE4s lifespan to fix this issue as they were fully aware of the issues in previous UE versions which was extremely similar just with the cylinder instead of the capsule. I understand why these classes are linked together but why they are core classes is what I never understood, why that functionality wasnt dropped further down into children so it was more readily overridable.

It might not be avoidable to create a new code path for the new system for backwards compatibility, issue is that creates something else to maintain which is why Im very cautious in the planning phase and it sounds like you took the time to outline a more modular system from the get go which does save alot of headaches like this in the future. There is always the possibility of not supporting those older code paths in newer engine versions and thats what I would personally prefer because I support any change which improves the engine :cool:

Hopefully we’ll see some actual traction on this.

Yeah definitely. In my ideal world it would be something of a priority but not holding my breath.

@JamesG did anything ever come of your discussions? Are we likely to hear about anything being started sometime this year?

This new roadmap also puts us in the dark more than ever.

I’m going to look at submitting some code soon, that splits up the movement component into somewhat reuseable classes, or at least breaks it into more manageable sections.

I’ve written CMC-like components for two games now, but I think I’ve made some nice changes to the overall layout.

TheJamsh: did your push your pull request(s)? Does your code make it possible to handle more versatile collision shapes than the vertical capsule component?

JamesG: any update from Epic?

I don’t have / didn’t make any pull requests, I wrote my custom components from scratch but it took a while to get a setup I’m happy with. I’m not fully settled on my design yet, but in theory you can have whatever collision shape you want.

However, you still have to write a huge proportion of the movement regardless of the collision shape, but that’s something you’ll have to expect really.

Hmmmm, another year almost bites the dust.

Epic probably aren’t going to do this unless they need it tbh, it’s a really massive change.

Doing so would also break thousands of projects unless they could create completely new code and seamlessly transition over.

Recommending to stay on a version of the engine so you don’t break your project and then not making much needed changes to the engine because it might break projects seems a little backwards to me. I can understand wanting the editor to actually open after without crashing out immediately but if you can get in to fix things to work with the hopefully documented changes I don’t see the problem. The road can’t always be a smooth ride, I know possession needs changes too.

4.19 is going to have some network improvements from fortenite: battle royale.

Characters are a pretty integral part of the engine - forcing everyone (including AAA where big $$$'s are spent by upgrading) to move to a completely rewritten character movement component is a big deal.

Even so, Epic aren’t going to take on a refactor of this scale unless they need it internally. The same is true to rendering features.

Which brings me back to what I’m always saying, deprecate character/cmc and make a predicted pawn & movementcomponent, then derive predicted pawn and build a much better character/cmc that actually meets standards.

Really feels like the only solution in my eyes.

The advantages:

  • You break nothing
  • You have no limitations from backwards compatability
  • Results in a predicted pawn (not only character) which is something the engine very much needs

Frankly I don’t see the purpose in continuing with Character/CMC.