That looks like it is supposed to at least. Does it work with other OpenXR applications?
@Rectus_SA as I did not know which other apps may use OpenXR I ended up trying The Blue (VR game) and it allowed me to acces the menu dashboard and I guess it worked as expected (as settings changes were having an effect on the look of the passtrhough)
So this is a temporary relief, but now the question arise of what could be happening that the DefaultVR Template from UE with those changes from Varjo does not effectively enable and use xr.OpenXREnvironmentBlendMode
I check the log out of curiosity, and it is pretty much identical to the previous one…
Looking at it, there seems to be an issue with the passthrough rendering in UE 5.4. It seems to render to a different swapchain image than the one actually being presented. I’ll have to look into why that is happening. Sorry!
Okay, thank you so much for your time!
PS: In this case I was actually trying with UE5.3, but maybe is something recent too
The issue seems to be that I wasn’t handling multiple images being open simultaneously in the same OpenXR swapchain (which is allowed by the spec). I guess Epic must have improved on the pipelining starting from 5.3.
I pushed an update to the layer, see if it works:
https://github.com/Rectus/openxr-steamvr-passthrough/releases/tag/v0.2.6-beta
Hi @Rectus_SA sorry for gettin back after a few days. I was able to try it and it really worked! That’s amazing, great job! I tested it with the valve index and it worked like a charm basically out of the box. Then I tried using it with the HTC Vive Pro and 2 but I got the issue that for some reason it is overlapping or weird mixing the images feed from both cameras, resulting in just white/light colors. Any idea or setting that should fix this?
1 Like
Unfortunately I haven’t been able to test with any of the Vive Pros, and there is no documentation from either Valve or HTC on how the cameras need to be set up. Does it happen with all the three projection modes (in the settings main page)?
I’d also appreciate any screenshots of the issue.
Yeah this situation happened for all 3 main projection modes and I tried playing around with the rest of the setting also. Here I attach to screenshots of how it looks the view when projection mode was 2D room view on top of the application TheBlue, in the latest one I discovered that it looks that its mainly taking the right camera on top of the left one (as that’s how it looked when I covered it)
The camera test from steamvr passed so no hardware issue on that regard
Here you can see the right one is covered
Thanks. The SteamVR camera test only shows its own output though. I would like to see what the actual in-game passthrough output looks like, ideally from the SteamVR VR View set to Both Eyes.
I’d also appreciate if you could take screenshots of the device properties of the Vive Pro’s from the passthrough settings menu. In the Debug tab of the menu click the Refresh button, and select the headset from the dropdown. It should show about two screens worth of parameters.
Thanks for the swift reply!
Here I attach the screenshots for both eyes from the device passthrough on the VRTemplate project I was using (both cameras uncovered)
And here is the debug output log within the layer app in the headset (Vive Pro 2)
1 Like
Thanks. I’ll have to dive into it and see if I can find the issue.
1 Like
I released an update. There were several issues with the vertical frame setup, and my code was assuming that the frame size was identical with the different distortion modes. I managed to get all the projection modes to output something that at least looks correct.
https://github.com/Rectus/openxr-steamvr-passthrough/releases/tag/v0.2.7-beta
Hello, sorry to revive the thread. I read through multiple times and tried to wrap my head around everything. I am testing on an HTC vive pro 2 and with the post process material in UE5.4. And it is working as expected with the 2D room view but when I switch to custom or 3D stereo I cannot see anything from the passthrough. I might be missing some simple thing but I would appreciate any insight on this.
Thank you for the amazing work though!
1 Like
I still haven’t been able to test it on real Vive Pro hardware. It’s likely another part of my custom projection code that still is broken. I’ll have another go at looking for issues.
It’s great that the 2D Room View mode works though. Thanks for testing it out and for reporting the issue!
One thing that would help is if you could check that all the device properties in the screenshots above matches yours, in case there are different hardware revisions with varying camera setups. Also if there are any error messages in the log.
Thanks for the quick reply. I took some screenshots of the device properties and the log as well
Only difference I could find compared to the screenshots above is with the individual cameras, most notibly in the intrinsics. But take a look, you understand it more than me.
Thanks! Everything looks normal, the differences must be from factory calibration.
I think I have figured out what the issue is. I’m using a fisheye lens model for undistorting SteamVR cameras, but the Vive cameras have non-fisheye lenses. I’m getting plausible results using the regular lens model, but since the Vive Pro 2 doesn’t provide any distortion coefficients, it’s still possible the output won’t be correct.
I’m going to release an update with the fix, as well as an override to force a different lens model. The update has a lot of other changes as well, so I need some time for testing it.
Awesome, thank you for the amazing work! If I still have access to the headset when you roll the update around I will test it!
I got it done: https://github.com/Rectus/openxr-steamvr-passthrough/releases/tag/v0.3.3-beta
If it doesn’t work, I’d like to see what the input camera frames look like. I added a button to save a frame to file in the debug tab, it will put it in %LOCALAPPDATA%
It does work. It does display passthrough in the other 2 modes now as well. Idk if it’s how it’s supposed to work but the custom 2d view and the 3d stereo view look the same and they both feel more zoomed in compared to the steamvr view and it’s a bit hard to perceive depth. As if I only had one eye open. Tried to grab some stuff on my table meanwhile and missed them for the first time. It feels much more natural in the default steam vr view mode. Made me a little nauseaous. But hey it’s no longer dark in the other two views, thank you for the quick update!