OpenXR HMD Plugin, Timing of xrWaitFrame Call

Development Pipeline & Plugins
, ,
1

I am using the OpenXR HDM Plugin with the Meta Horizon App OpenXR Runtime and Meta Quest Pro. I am trying to understand the placement of the xrWaitFrame() call, and why it is currently located in the OnStartGameFrame() call. I am running a game that can implement a variable game tick delta, however, by placing the xrWaitFrame() in OnStartGameFrame(), it prevents me from initiating the world tick with the best estimate of the DeltaTime I should use. I understand there is note in the plugin that mentions placing xrWaitFrame() earlier could cause the pipeline to stall. So I guess I have a couple questions:

  1. Would using the variable time delta with my simulation even improve the smoothness of the simulation? I know there is some late update steps and time warping, do these take care of any issue of a simulation time being misaligned with the display time?

  2. Has anyone played around with placing the xrWaitFrame() call earlier, before UpdateTimeAndHandleMaxTickRate. Where exactly did you put it and did it work well.

  3. If I don’t move the xrWaitFrame, is there a smart way to provide a small “late update” to the tick delta right after xrWaitFrame() returns? My AI coding agent said maybe, but that there were are other things that get triggered in the FEngineLoop before UWorld_Tick that will essentially be stuck with the old delta time.

As I understand the relevant game-thread order, it is approximately:

FEngineLoop::Tick
  → UEngine::UpdateTimeAndHandleMaxTickRate
      → sets FApp::DeltaTime
  → GEngine->Tick(FApp::GetDeltaTime())
  → UGameEngine::Tick(DeltaSeconds)
  → UWorld::Tick(LEVELTICK_All, DeltaSeconds)
      → XRSystem->OnStartGameFrame(...)
          → FOpenXRHMD::OnStartGameFrame(...)
          → FOpenXRHMD::OnBeginSimulation_GameThread()
          → xrWaitFrame(...)
              → returns predictedDisplayTime
      → apply time dilation and FixupDeltaSeconds
      → assign UWorld::DeltaTimeSeconds
      → networking, physics, actors and components

I noticed the following comment in the UE5 OpenXR implementation, but I wished there was a little more context or guidance around it:

// TODO: We could do this earlier in the pipeline and allow simulation
// to run one frame ahead of the render thread.
// That would allow us to take more advantage of Late Update and give
// projects more headroom for simulation.
// However currently blocking in earlier callbacks can result in a
// pipeline stall, so we do it here instead.