Announcement

Collapse
No announcement yet.

Movement Replication freezes on Clients in AR Shared World Multiplayer (IOS, LanSession)

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Movement Replication freezes on Clients in AR Shared World Multiplayer (IOS, LanSession)

    Hi, we have a problem and hope to find some help in the community.


    We are working with AR Shared World classes and local Lan-Session on iOS.
    Problem is that our replicated Actors freeze on clients and do not move for around 2 minutes until movement replication suddenly starts.

    Any help is greatly appreciated.


    Our worksflow:

    We set up AR World Map synchronization (shared AR world) according to this article:
    „A preview of persistent and shared AR experiences in UE4 4.21“ on Medium from Joe Graf. Link: https://medium.com/@joe.j.graf/a-pre...1-c47b3331a4b1

    Since this preview from october 2018 no form of documentation, content example or tutorial ever followed, so we had to guess, check source code and try to get things working.

    Using ARSharedWorldGameMode, ARSharedWorldGameState and ARSharedWorldPlayerController.
    Recap how these classes work:
    • Listen server saves it‘s AR world data (an array of bytes. this is also a reason why a dedicated server makes no sense, as it would not be able to gather AR data)
    • Listen server creates an online session (isLan true, 4 connections)
    • Client finds online session and joins.
    • at this point the ARSharedWorldGameMode sends it‘s AR data in chunks over several ticks to the client. It saves the AR data in the ARSharedWorldGameState. Size depends on tracked area and can be around 2-3MB + a preview jpeg image around 300kb. An UI-Widget and print string do confirm the successful transfer process.
    • after this AR data was successfully transfered, we start an AR session with this data on the client. ARKit takes care of rebasing the client’s ar session and so both have the same world origin. This AR relocalisation is a one time process when a client joins a session.
    • so far no problem and from now on both player have the same ue4 world origin and the rest should be normal ue4 replication as with any other multiplayer game.

    After the client connected to the server and the Server spawned a Character, we can only see movement of characters and actors on the server.
    On the client the replicated actors remain still at there spawn transform.
    It takes several minutes of waiting until movement finally also replicates to clients.
    After this mysterious waiting time, that we cannot explain, everything is replicated/synchronized pretty well.
    • No matter if the character is controlled by server or client.
    • During this freeze time, physics simulations also do not replicate. Only skeletal animation (idle) can be seen on clients.
    • all our RPC-input events on the client device are successfully called on the server and work as expected on the server. Only movement replication seems to be problematic.
    • Stat commands show no hitches in frame rate, cpu/gpu or networking. Meshes have together only few thousand polys.
    • All these actors have „net priority“ set to 3, „net update frequency“ to 100 and „always relevant“ to true.
    • For all actors and characters we tried to replicate movement through ue4 actor settings (replicates+replicate movement) and also SmoothSync Plugin. But in both cases the problem persists.
    • We are spawning characters after the client joined the session and both AR worlds finished synchronizing.

    Each player controls a character to move. The characters are not possessed, because the AR Pawn that holds the AR camera needs to be possesed of course (it is set to not replicate). Inputs for Characters are sent via RPCs to the server, to add server authoritive movement to the character.

    Maybe less important, but still mentioning: On each character are 3 physics simulated actors attached, whose movements are replicated too. But a test with a new project, the standard UE4 mannequin character and nothing attached gave the exact same result of a long pause before any movement replication starts.


    This only happens when the function „SetARWorldSharingIsReady“ in the ARSharedWorldGameMode is used.
    If we don’t call this function, then no AR data will be delivered from server to client and the movement of actors is instantly replicated as expected.


    This leads us to believe that there is something going on in these ARSharedWorld classes rearding networking.
    https://github.com/EpicGames/UnrealE...eality/Private


    Connection is saturated for several minutes after AR Shared World data was sent.
    Using stat net, on the server the “saturated“ stays at 1 during the transfer of AR data and the initial movement freeze, then changes to 0 when movement replication begins. We wonder why the connection remains saturated even after all the AR data was delivered?

    Other stats can be seen in the attached screenshots.


    Version info

    Engine Version: 4.22.3, Xcode 10.1, MacOS Catalina
    fresh HandheldAR template. No unusual project settings.

    tested devices: iPad Pro 2018 11“, iPad 2017, iPhone7, all run IOS 13.
    Same wifi 5ghz, allowed by router to open ports. No connection or speed issues otherwise with this wifi network.

    OnlineSession useLAN=true, OnlineSubsystem=NULL in DefaultEngine.ini
    Always ListenServer, not dedicated Server.
    Also tried deactivating online subsystem ios plugin, no difference.


    Thank you very much for taking the time to read!

    Click image for larger version  Name:	ServerARTransfer.jpeg Views:	7 Size:	301.9 KB ID:	1720787Click image for larger version  Name:	ServerNoReplication.jpeg Views:	7 Size:	304.4 KB ID:	1720788Click image for larger version  Name:	ServerReplicated.jpeg Views:	7 Size:	303.4 KB ID:	1720789Click image for larger version  Name:	ClientNoReplication.jpeg Views:	8 Size:	233.3 KB ID:	1720785Click image for larger version  Name:	ClientReplicated.jpeg Views:	7 Size:	224.4 KB ID:	1720786
    Last edited by NobbigvonL; 02-13-2020, 12:57 PM.

    #2
    Hello.
    I have the same problem.
    ARWorldData has been successfully synchronized, but the actor cannot be duplicated afterwards.
    I think ARWorldData and PreviewImage need to be sent in some other way, but have you found a solution ever?

    Comment


      #3
      Hello,
      no, sadly we haven´t found any solution. We have checked the ARSharedWorld classes in source code, but still no idea.

      Comment


        #4
        Code:
        void SetARWorldSharingIsReady()
        This function may help.

        I am also finding AARSharedWorldGameMode::Tick is a private function member, so how can it be called by the engine? By reflection system?
        Last edited by svandex; 10-20-2020, 03:40 AM.

        Comment


          #5
          Thanks for the reply, we used blueprints, called SetARWorldSharingIsReady and assume it succeeded, as the AR sharing process worked fine.
          It's only Actor Movement (and skeletal mesh animation) that is blocked, after ARSharing finished.

          We made some process with two steps:

          1.
          We gave command line arguments when the server opens the multiplayer player map: -bIsLanMatch and -lanplay

          Documentation says: -LANPLAY "Tell the engine to not cap client bandwidth when connecting to servers. Causes double the amount of server updates and can saturate client's bandwidth." https://docs.unrealengine.com/en-US/...nts/index.html


          2.
          Changed settings in the DefaultEngine.ini to higher values.

          [/Script/Engine.Player]
          ConfiguredInternetSpeed=30000
          ConfiguredLanSpeed=30000


          [/Script/OnlineSubsystemUtils.IpNetDriver]
          MaxClientRate=30000
          MaxInternetClientRate=30000

          and probably some other settings i currently don't recall to higher values.


          (Internetspeed AND LanSpeed to high values because an older forums thread reports the Engine did not use LanSpeed in lan mode, so just to be sure we changed both values.)

          This seems to work at first glance, but we are not sure if there are any downsides to this. Waiting for 4.26 before further testing.



          svandex
          For you problem with AARSharedWorldGameMode::Tick, try a C++ post or the discord server https://unrealslackers.org/ , Good Luck

          Comment

          Working...
          X