That looks like it is supposed to at least. Does it work with other OpenXR applications?
@Rectus_SA as I did not know which other apps may use OpenXR I ended up trying The Blue (VR game) and it allowed me to acces the menu dashboard and I guess it worked as expected (as settings changes were having an effect on the look of the passtrhough)
So this is a temporary relief, but now the question arise of what could be happening that the DefaultVR Template from UE with those changes from Varjo does not effectively enable and use xr.OpenXREnvironmentBlendMode
I check the log out of curiosity, and it is pretty much identical to the previous oneā¦
Looking at it, there seems to be an issue with the passthrough rendering in UE 5.4. It seems to render to a different swapchain image than the one actually being presented. Iāll have to look into why that is happening. Sorry!
Okay, thank you so much for your time!
PS: In this case I was actually trying with UE5.3, but maybe is something recent too
The issue seems to be that I wasnāt handling multiple images being open simultaneously in the same OpenXR swapchain (which is allowed by the spec). I guess Epic must have improved on the pipelining starting from 5.3.
I pushed an update to the layer, see if it works:
https://github.com/Rectus/openxr-steamvr-passthrough/releases/tag/v0.2.6-beta
Hi @Rectus_SA sorry for gettin back after a few days. I was able to try it and it really worked! Thatās amazing, great job! I tested it with the valve index and it worked like a charm basically out of the box. Then I tried using it with the HTC Vive Pro and 2 but I got the issue that for some reason it is overlapping or weird mixing the images feed from both cameras, resulting in just white/light colors. Any idea or setting that should fix this?
Unfortunately I havenāt been able to test with any of the Vive Pros, and there is no documentation from either Valve or HTC on how the cameras need to be set up. Does it happen with all the three projection modes (in the settings main page)?
Iād also appreciate any screenshots of the issue.
Yeah this situation happened for all 3 main projection modes and I tried playing around with the rest of the setting also. Here I attach to screenshots of how it looks the view when projection mode was 2D room view on top of the application TheBlue, in the latest one I discovered that it looks that its mainly taking the right camera on top of the left one (as thatās how it looked when I covered it)
The camera test from steamvr passed so no hardware issue on that regard
Here you can see the right one is covered
Thanks. The SteamVR camera test only shows its own output though. I would like to see what the actual in-game passthrough output looks like, ideally from the SteamVR VR View set to Both Eyes.
Iād also appreciate if you could take screenshots of the device properties of the Vive Proās from the passthrough settings menu. In the Debug tab of the menu click the Refresh button, and select the headset from the dropdown. It should show about two screens worth of parameters.
Thanks. Iāll have to dive into it and see if I can find the issue.
I released an update. There were several issues with the vertical frame setup, and my code was assuming that the frame size was identical with the different distortion modes. I managed to get all the projection modes to output something that at least looks correct.
https://github.com/Rectus/openxr-steamvr-passthrough/releases/tag/v0.2.7-beta
Hello, sorry to revive the thread. I read through multiple times and tried to wrap my head around everything. I am testing on an HTC vive pro 2 and with the post process material in UE5.4. And it is working as expected with the 2D room view but when I switch to custom or 3D stereo I cannot see anything from the passthrough. I might be missing some simple thing but I would appreciate any insight on this.
Thank you for the amazing work though!
I still havenāt been able to test it on real Vive Pro hardware. Itās likely another part of my custom projection code that still is broken. Iāll have another go at looking for issues.
Itās great that the 2D Room View mode works though. Thanks for testing it out and for reporting the issue!
One thing that would help is if you could check that all the device properties in the screenshots above matches yours, in case there are different hardware revisions with varying camera setups. Also if there are any error messages in the log.
Thanks! Everything looks normal, the differences must be from factory calibration.
I think I have figured out what the issue is. Iām using a fisheye lens model for undistorting SteamVR cameras, but the Vive cameras have non-fisheye lenses. Iām getting plausible results using the regular lens model, but since the Vive Pro 2 doesnāt provide any distortion coefficients, itās still possible the output wonāt be correct.
Iām going to release an update with the fix, as well as an override to force a different lens model. The update has a lot of other changes as well, so I need some time for testing it.
Awesome, thank you for the amazing work! If I still have access to the headset when you roll the update around I will test it!
I got it done: https://github.com/Rectus/openxr-steamvr-passthrough/releases/tag/v0.3.3-beta
If it doesnāt work, Iād like to see what the input camera frames look like. I added a button to save a frame to file in the debug tab, it will put it in %LOCALAPPDATA%
It does work. It does display passthrough in the other 2 modes now as well. Idk if itās how itās supposed to work but the custom 2d view and the 3d stereo view look the same and they both feel more zoomed in compared to the steamvr view and itās a bit hard to perceive depth. As if I only had one eye open. Tried to grab some stuff on my table meanwhile and missed them for the first time. It feels much more natural in the default steam vr view mode. Made me a little nauseaous. But hey itās no longer dark in the other two views, thank you for the quick update!