Having the same exact problem right now! I also need to have a desktop UI which offers a selection mode to the user. On top of that, the user needs to be able to hit a button to exit VR, and hit a button on desktop to go back into VR.
I have this functionality working in Standalone mode from Editor. However, a packaged game will only ever work for pancake (not VR). Not sure if I actually have to push the build to Oculus in order for VR to work, even though the app shows as “running” in Oculus Home.