how to get VR pawn to work with elevator?

Did you maybe forget that your VR HMD Headset is just a Camera being relatively offset of your actual pawn?
The Actor Location of your Pawn does NOT have to be the place you are standing. The Actor Location is the middle of your VR bounds.