# HTC Vive passthrough camera AR/VR development

**URL:** <https://forums.unrealengine.com/t/htc-vive-passthrough-camera-ar-vr-development/1186330>\
**Category:** XR Development\
**Tags:** unreal-engine, question\
**Created:** [June 2, 2023, 3:04am UTC](https://forums.unrealengine.com/t/htc-vive-passthrough-camera-ar-vr-development/1186330 "2023-06-02T03:04:38Z")\
**Posts on this page:** 5\
**Page:** 3

<div class="post-metadata">

**Author:** ![Rectus\_SA](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@Rectus\_SA](https://forums.unrealengine.com/u/Rectus_SA)\
**Post date:** [April 7, 2025, 11:18am UTC](https://forums.unrealengine.com/t/htc-vive-passthrough-camera-ar-vr-development/1186330/41 "2025-04-07T11:18:53Z")

</div>

Thanks for testing it!

The 2D Custom mode looking worse would mean it’s not able to properly rectify the image. Both modes looking the same likely means it’s not able to calculate depth and falling back to the 2D projection.

The headset doesn’t seem to provide the camera distortion parameters to correctly calculate the rectification. It would be up to HTC (and possibly Valve) to update the firmware and drivers to provide the correct parameters. Not sure that is possible without factory calibration either.

I could try adding support for using the stock pre-rectified camera frames for 3D reconstruction, but it doesn’t work well for the Index at least.

---

<div class="post-metadata">

**Author:** ![unityafise](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@unityafise](https://forums.unrealengine.com/u/unityafise)\
**Post date:** [February 4, 2026, 9:19am UTC](https://forums.unrealengine.com/t/htc-vive-passthrough-camera-ar-vr-development/1186330/42 "2026-02-04T09:19:36Z")

</div>

Hey, how can I enable VST in my game? I am using the **Varjo XR-4**. I have enabled **MR** and everything else works fine. The **alpha blend** works perfectly in **Varjo Base** , but I think the VST layer is applied **after** the Unreal render is complete. Hence, it shows black where alpha blend is enabled. I want it to show the real-world feed wherever the alpha blend is enabled. I am trying to achieve an MR world, which I have, but it only shows the real-world feed in **Varjo Base** , not in the editor or game itself.

---

<div class="post-metadata">

**Author:** ![Rectus\_SA](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@Rectus\_SA](https://forums.unrealengine.com/u/Rectus_SA)\
**Post date:** [February 4, 2026, 12:39pm UTC](https://forums.unrealengine.com/t/htc-vive-passthrough-camera-ar-vr-development/1186330/43 "2026-02-04T12:39:45Z")

</div>

Varjo has native support for passthrough when you are using their own OpenXR runtime, so make sure you are using that instead of SteamVR. My applications are only for when using SteamVR, and I don’t know if they work with Varjo headsets, since there is no documentation on whether the camera feed is available through SteamVR.

As you say, the passthrough applied after the render inside the runtime compositor. Unreal renders scenes using an inverted alpha channel. When using unrendered parts of the scene as the source of the cutouts, they will end up inverted by default. Unreal 5.7 added support for the new `XR_EXT_composition_layer_inverted_alpha` OpenXR extension that flags to the runtime to treat the alpha as inverted. I don’t know if the Varjo runtime supports it yet.

You can enable it by enabling `“Invert scene alpha for passthrough"`in the Project settings for the OpenXR plugin, and checking if it’s working with the `“Is Composition Layer Inverted Alpha Enabled”` blueprint node.

If the runtime doesn’t support it (or you are using an older Unreal version), you will have to add a postprocess material to manually invert the alpha. Varjo has a tutorial on how to set it up (you don’t need the Varjo plugin for 5.7 unless you want to use additional features like depth composition):

> **[Mixed Reality with Unreal Engine 5](https://developer.varjo.com/docs/v4.6.0/unreal/ue5/mixed-reality-with-unreal5)**
>
> Developing mixed reality applications for Varjo XR headsets is quick and easy with Varjo OpenXR plugin. Follow these instructions to get started.

If you want to support other runtimes as well, you can either always have the postprocess material on, or use the output of the `“Is Composition Layer Inverted Alpha Enabled”` node to toggle it off.

Epic has a bit more documentation on the 5.7 passthrough feature here:

[https://portal.productboard.com/epicgames/1-unreal-engine-public-roadmap/c/2040-openxr-passthrough](https://portal.productboard.com/epicgames/1-unreal-engine-public-roadmap/c/2040-openxr-passthrough)

---

<div class="post-metadata">

**Author:** ![unityafise](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@unityafise](https://forums.unrealengine.com/u/unityafise)\
**Post date:** [February 6, 2026, 4:57am UTC](https://forums.unrealengine.com/t/htc-vive-passthrough-camera-ar-vr-development/1186330/44 "2026-02-06T04:57:49Z")

</div>

Hi, thanks for the reply, but my issue is different.

 ![varjomask](https://d3kjluh73b9h9o.cloudfront.net/original/4X/3/d/8/3d84cd479b95c0328d4bade5d2ec53166fe45f03.jpeg)

 ![varjomaskvst](https://d3kjluh73b9h9o.cloudfront.net/original/4X/7/9/8/798faca64825b496f7673d4d7d89322347f55d63.jpeg)

I already have passthrough (VST) working correctly **inside the HMD** using Varjo OpenXR (alpha blend / MR).

The problem is the **desktop game window / spectator screen** : wherever my scene outputs alpha (cutout/mask), the monitor view shows **black** , while the HMD shows the real-world camera feed (as expected).

I assume this is because the VST video is composited by the **Varjo runtime compositor** _after_ Unreal renders, so Unreal’s desktop backbuffer never contains the camera pixels. (Attached screenshots: Varjo headset view vs Unreal window).

My question: **Is it possible to show the actual VST camera feed in the Unreal spectator/monitor output as well (i.e., have the composited passthrough visible on the desktop), or does the app need access to the camera frames as a texture to composite it itself?**

If it’s not possible with OpenXR passthrough, what’s the recommended approach?

---

<div class="post-metadata">

**Author:** ![Rectus\_SA](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@Rectus\_SA](https://forums.unrealengine.com/u/Rectus_SA)\
**Post date:** [February 6, 2026, 2:45pm UTC](https://forums.unrealengine.com/t/htc-vive-passthrough-camera-ar-vr-development/1186330/45 "2026-02-06T14:45:17Z")

</div>

The app would need access to the camera frame textures.

It’s not possible to do with any passthrough method provided by OpenXR itself. They are all designed to only allow the compositor access to the camera frames. This is done for several reasons. Mainly for keeping the camera latency low, but also for privacy and simplicity.

I don’t think there are any official solutions for compositing the cameras in-engine. My old `UE4SteamVRPassthrough` plugin does it, but it only works for HMDs that expose the camera streams to the SteamVR driver API, and even then only without stereo-correct projection. I highly doubt Varjo has added support for this. Valve never properly published the driver-side API, and its practically hardcoded to only work with their first-party HMDs.

Varjo does seem to provide access to the raw distorted camera frames in their own native SDK at least. See the `Varjo_datastream.h` and `Varjo_types_datastream.h`headers. The headers suggest you would need a Varjo Base Pro license to access the streams though.

[https://developer.varjo.com/docs/native/varjo-native-sdk](https://developer.varjo.com/docs/native/varjo-native-sdk)

You would have to write a plugin to manually undistort the frames, optionally perform stereo estimation, and then reproject and composite the frames.

[Previous page](https://forums.unrealengine.com/t/htc-vive-passthrough-camera-ar-vr-development/1186330.md?page=2)
