Help with Replicating VR Avatar Movements and Hand Tracking in Unreal Engine 5.5

Development Programming & Scripting Multiplayer & Networking
, ,
1

Dear Unreal Engine Support Team,

I am new to Unreal Engine, and I have recently started exploring VR development. I have already set up and successfully created clients and servers for my project. However, I am now looking to understand how to replicate VR avatar movements across multiple clients, including hand tracking functionality for the future.

To give you some context, I have a multiplayer project in Unreal Engine 5.5, and I want to ensure that the movements of VR avatars (e.g., their position, rotation, hand movements, and hand tracking) are visible and synchronized for all connected clients.

Here are a few specific questions I have:

  1. How can I replicate the position and rotation of VR avatars to all clients in a multiplayer environment?
  2. What steps should I follow to ensure that hand movements (controllers in real life) are replicated in the VR environment?
  3. For the future, how can I integrate hand tracking in Unreal Engine and replicate those hand movements across all clients?
  4. Are there any recommended best practices for VR replication in Unreal Engine 5.5, especially when it comes to handling player movements and interactions in VR?
  5. Do I need to use specific components or plugins for replicating VR avatars in multiplayer mode, or is this functionality built into Unreal Engine?

I would really appreciate your guidance on the best approach to take for this. Any pointers or resources you can provide will be extremely helpful as I continue learning Unreal Engine.

Thank you in advance for your help!

Best regards

2

Hi,

In terms of resources, we do have the Collab Viewer Template project, which is set up to support VR pawns in a multiplayer environment. However, while the blueprints there could be a helpful reference, this template was designed for enterprise applications, and the approach used for replicating the pawns is not recommended for a multiplayer game.

There, the pawn’s position and movement is client-authoritative, with the server simply receiving updates from the pawn’s owning client and replicating the data to the others.

In order for the experience to feel good, it is necessary for the client to immediately respond to player input and locally update its pawn. However, we always recommend that the server check and verify any data received from clients, in order to prevent cheating and exploits. This is also necessary so that the server can correct any erroneous client actions, which can occur due to packet loss, latency, or other game-specific interactions between clients. (For example, this is how the character movement component handles replicating movement for non-VR pawns.)

In terms of best practices, I’m not aware of anything specific to VR replication. The general best practices for networking and replication would apply, and you can find a number of resources on this both in our documentation and [knowledge base [Content removed]

Finally, for an off-the-shelf solution, we recommend checking out the VR Expansion Plugin: https://vreue4.com/features?section=multiplayer-support

This includes different options for handling VR replication, although it is likely that some project-specific handling will still be needed.

If you have any further questions, please don’t hesitate to reach out!

Thanks,

Alex

3

Hi Alex,

Thank you very much for your detailed and helpful response — I really appreciate the time you took to explain the replication approach and the potential pitfalls of client-authoritative movement.

Just to clarify the context a bit further: my project is not a competitive online game, but rather a location-based VR (LBVR) multiplayer experience, where users interact in the same physical space (like a museum or themed attraction). Because of that, I’m a bit less concerned about cheating or malicious clients and more focused on smooth and low-latency replication of head and hand movements to ensure presence and collision avoidance between users.

I’ll definitely look into the VR Expansion Plugin you recommended — it sounds like it could provide a solid foundation for this kind of setup.

Thanks again for your support, and I may reach out again as I get deeper into the implementation!

Best regards,

Romain

4

Hi,

You’re very welcome, and we’re happy to answer any other questions you may have! Please note though that support will be limited for the next two weeks due to the company break.

Thanks,

Alex