Finally sorted this out.
I had a UI widget which i used to draw the crosshair/target for the GVR controller. For whatever reason it was interfering with the explicitly set CPU/GPU levels and was having quite a huge impact on performance. Replacing that with a quad Stereo Layer solved both the problems.