PCVR development with UE5 and Pico 4, performance drop after first preview

Development Platform & Builds XR Development
1

Setup info: RTX 3080ti, AMD Ryzen 9 5900X 12-Core, Pico 4 with SteamVR over USB3

Hi

I tried starting the VR template yesterday and had a lot of trouble finding the necessary information to get me started. I am also very new to VR in general.

The Pico documentation has a rather long walkthrough but is more focused on developing for standalone HMD. I would prefer to keep it as simple as possible and also prefer PCVR.

What are the minimum steps that have to be done in order to get going?

I downloaded the UE5 sdk for the Pico headhear (which is stated as experimental on their webpage) and put it in the template project’s Plugins folder. I started the streaming assistant and SteamVR and connected the HMD. The VR preview becomes available and when I run, the HMD display starts the preview and everything seems to work fine. Performance is stable and does not seem to degrade over time while in the first preview.

However there are a few things that bother me…
When I exit the app through the HMD the project is always closed on the PC. (idk if this is expected)
When I cancel the preview on PC it also correctly closes the preview in the HMD and keeps the project open, however, the HMD shows that the app is still running! (maybe in the background)

Now everytime I stop the first preview and start the preview again in UE5 the performance becomes worse, very quickly up to a non-playable state (insanely low FPS and responsiveness of inputs). The engine outputs an error message like “Video memory has been exhausted” with - in the worst situations up to 20 GB over budget, but it normally quickly reduces to 1-2GB after stopping preview. This only happens after the first preview has stopped, which runs very smoothly!

Reconnecting the HMD does not help. I can only start the next preview normally after completely killing the SteamVR processes and then freshly connecting the HMD. Then I can again run one preview before the performance drops again.

Am I missing something essential?
Is there anything else that needs to be setup?

Do I really need Android Studio and all the other mobile settings if I actually plan to work with PCVR?

Does this seem more like a Pico relates problem or UE/SteamVR? To me it almost seems like SteamVR doesn’t close the session completely or has some memory leak and therefore the memory gets filled to the brim with every successive preview run.

Could Virtual Desktop help with performance? As the streaming performance per se is not that bad with the native Streaming Assistant software and just degrades over multipla playthroughs, I personally do not think this is the problem here…

Advice would be greatly appreciated!
Thanks

1 Like
2

So today I got some time to come back to it and try again.
I chose to invest the 20 bucks in Virtual Desktop as it’s peanuts anyways compared the Pico purchase ;D

And it seems Virtual Desktop actually helps A LOT when running UE 5.0 projects! (no advertisement intended)
The problem described above appeared too, but I think it highly depends on what programs you started how and when.

It might be that it works best to connect via Virtual Desktop, then start the project. UE 5 seems to note that a HMD is connected and automatically starts SteamVR. When I run the preview then, everything seems to work fine.

Also I can close the game via the ingame UI and also via Engine. The HMD is then sent to the SteamVR Home. Steam VR should not be closed.

I will test further and get back to this thread if I find anything new. I will also try 5.1 compatibility.

EDIT: I tested the native Streaming Assistant again and have to admit, the problem of restarting might have gone with a version update at some point. You can still crank the settings to a point where the HMD is just completely overwhelmed, but this will start with ingame performance problems and might force you to restart SteamVR and the Engine. The original issue had no initial ingame performance issues whatsoever, they only came in successive preview plays.

Still the Virtual Desktop has a lot more control over streaming quality and bitrate, so I think anyone who might be reading this should still consider the purchase.

UE5.1 works too but I was not able to recompile the plugins. So , as I am running this via PCVR I tested if the plugins were actually necessary. And they are not!!!
So the Pico SDK and generally Pico Plugins for UE seem to be only necessary for developing for Pico standalone. This is kinda expectable, but the Pico documentation on PCVR dev is a bit meagre.

So to anyone who’s interested, Pico 4 is valid for UE5.1 VR development. You have to play a bit with the settings and I recommend using Virtual Desktop to be able to fine tune the settings.

I tried different settings and the one I tried last that had nice quality and decent performance was:

  • SteamVR resolution at 200% (150 or Auto is ok too)
  • Virtual Desktop Quality at Godlike (less is better for performance and battery but it works)
  • Virtual Desktop Bitrate at 30-60 (less is better for performance)

Cheers!

3

You do not need any android build tools at all if you are only doing pcvr
just select windows64 as the target platform only.
I only use the openxr and hp plugins, no steam or oculous plugs, even when I build for quest.
Some projects I use VR expansion plugin template as the base, others I use the open xr ue template.

I doubt you will actually “need” the pico plugins if you decide to build for quest
everything has switched to open xr, even the pico.

if you do decide to build for quest, you need about 6 separate things installed correctly
jdk8, jre8, android studio 4, dotnet 3.5, sdk29, ndk 21…
the memory jam might be caused by using the virtual desktop app for the hmd preview.
you should be able to pcvr test without it.

4

dotnet? do I install this with visual studio 2019 because I didnt install this. Also I saw people with sdk 30 that work. I have problem default unreal project is packed but i stuck in loading screen on VR Pico 4 although I hear fire sound from app. Also its impossible to exit from that loading screen I have to reset Pico. Do you have Pico 4 and can you help me with this?

5

the dotnet 3.5 is the redist runtime
the UAT tool in unreal uses it.

Yes sdk 30 works.
29 or 30.
ndk is 25 now
previously was 21.

I don’t have a pico.

but you should try to not use virtual destop and just use the vr preview in unreal
it is open xr and should work
are you able to preview the level in vr while in the editor?
If not then you have to make that go first.

Also
If you eventually pkg an apk for the pico
it can’t use ogl, only vulkan.
or you will get a black screen

6

there was a problem with plugin subonline something like that, when I turn off that it packed ok and it works, have to do some tweeks but it work!

7

hi people. do you have any guide for pico 4 in unreal?. i put Pico_pAwn and when i start the game. i doesnt start (controllers and chamera).