Second VR player joining multiplayer session has black screen in their headset

I’ve been following the official Unreal Engine blueprint multiplayer tutorial on YouTube to help me setup multiplayer in Unreal Engine 5.1.1. Everything works as expected when using a regular character pawn which just consists of a cube with a camera and a basic movement blueprint. But when I try using a VR character, the VR Pawn from the VR Template or something I’ve made myself, the second player that joins the session, will always have a black screen in their headset. I’ve tried with multiple computers as well as the default network debugging functions in engine, but both yield the same results.

From what it looks like, it feels like the HMD isn’t getting initialized properly, or not at all, for the second player.

It might be worth noting that the game view on the monitor is not black, only in the headset.

I feel like I’ve looked everywhere, but nobody has mentioned having this issue so I’m not sure what I’m doing wrong.

Any suggestions? Or ideas for what’s going wrong?