Panoramic Rendering and mismatched viewer scale in the 360 degree video

I’m trying to render out a scene using the Panoramic Rendering option for a 360 degree video which will be viewed in a headset. The render is successful, however I’m observing that the viewer’s perspective is off. The size of the viewer is very reduced and the world appears twice as big in the video when viewed in the VR headset. How can this imbalance be fixed?