Integrating OpenXR Tracker Data with nDisplay for CAVE Setup – Issues When Launching via Switchboard

Development Programming & Scripting
,
1

Hi all,

I’m working with an nDisplay-based CAVE setup and aiming to extend its functionality to support Unreal Engine VR projects out of the box.

To achieve this, I developed an OpenXR API layer that receives tracker data from Motive (using “Glasses” to represent head position) and passes it to Unreal as the VR HMD position. I’m using SteamVR in headless mode as the OpenXR runtime—so the system runs without needing a headset connected.

Project Setup:

  • Engine: Unreal Engine 5.5
  • Project: VR Template project with both a VRPawn and an nDisplay config.
  • Tracking Logic: In the Level Blueprint, every tick, I set the nDisplay config’s location to match the VRPawn’s camera component location (which comes from my OpenXR layer).

Behavior Observed:

  • In VR Preview: Works fine — the camera updates correctly from the OpenXR data.
  • Packaged EXE (launched manually): Also works as expected — camera and nDisplay config follow OpenXR data.
  • Packaged EXE via Switchboard: Crashes on startup with a black screen

Debugging via Command Line:

When launching the EXE with the same command line as Switchboard:

  • Removing -xrtrackingonly: App runs, but no tracking data appears in the OpenXR layer logs.
  • Removing -noxrstereo: App crashes with a black screen; again, no tracking data is logged.
  • Keeping both (-xrtrackingonly and -noxrstereo): App does not crash, and tracking data is logged by the OpenXR layer — but Unreal doesn’t seem to receive it.

My Goal:

To bridge the connection between OpenXR and nDisplay, such that:

  • The application launched via Switchboard (as is required for my CAVE display),
  • Receives tracker data from OpenXR, and
  • Applies it correctly in the nDisplay component

Key Questions:

  • Is this connection between OpenXR head tracking and nDisplay even possible in this scenario?
  • If so, what am I missing in terms of runtime flags, configuration, or code integration?
  • What are my options for such an implementation?

Current Launch Command:

.\OpenXR_VRPN_Test.exe -game -messaging -dc_cluster -nosplash -fixedseed -NoVerifyGC -noxrstereo -xrtrackingonly -RemoteControlIsHeadless -StageFriendlyName=“node_front_left_LE” -MaxGPUCount=2 -dc_cfg=“C:\Users<username>\AppData\Local\Temp\ndisplay\6736569449170DC2A33C709201D7D41B.ndisplay” -dx12 -dc_dev_mono -dc_node=node_front_left_LE Log=node_front_left_LE.log -ini:Engine:[/Script/Engine.Engine]:GameEngine=/Script/DisplayCluster.DisplayClusterGameEngine,[/Script/Engine.Engine]:GameViewportClientClassName=/Script/DisplayCluster.DisplayClusterViewportClient,[/Script/Engine.UserInterfaceSettings]:bAllowHighDPIInGameMode=True -ini:Game:[/Script/EngineSettings.GeneralProjectSettings]:bUseBorderlessWindow=True -ini:Input:[/Script/Engine.InputSettings]:DefaultPlayerInputClass=/Script/DisplayCluster.DisplayClusterPlayerInput -unattended -NoScreenMessages -handleensurepercent=0 -UDPMESSAGING_TRANSPORT_MULTICAST=“230.0.0.1:6666” -UDPMESSAGING_TRANSPORT_UNICAST=“192.168.10.4:0” -UDPMESSAGING_TRANSPORT_STATIC=“169.254.3.1:9030” -ExecCmds=“DisableAllScreenMessages” -fullscreen -CONCERTRETRYAUTOCONNECTONERROR -CONCERTAUTOCONNECT -CONCERTSERVER=“ai4coping_forest_MU_Server” -CONCERTSESSION=“MU_Session” -CONCERTDISPLAYNAME=“node_front_left_LE” -CONCERTISHEADLESS -DPCVars=“Slate.bAllowNotifications=0,p.Chaos.Solver.Deterministic=1”

Any insights or experiences from others working with OpenXR, nDisplay, and Switchboard would be highly appreciated!

Thanks in advance.