Can't run VR app as a dedicated server

I have a VR app that I’m building multiplayer for.

When I run the game it works fine in VR preview. However If i run the game with dedicated server and use VR preview or PIE or Standalone game then it spawns the headset in some arbitrary place far away from the actual spawn. It also doesn’t spawn into the VR pawn i created.