weird dark outlines when video playing in Quest2

It’s something that is done in code where the engine interfaces with the VR runtime.

I looked at the code, and it seems for the Oculus plugin, it actually applies the masking in the engine instead of in the runtime. To fix it, you would need to modify the engine, specifically, the FLayer::UpdateTexture_RenderThread() function in OculusHMD_Layer.cpp, and add a way to set bAlphaPremultiply to false for your layer.

For completions sake, the OpenXR plugin does it on the runtime side by passing the XR_COMPOSITION_LAYER_UNPREMULTIPLIED_ALPHA_BIT flag when creating a quad composition layer. Toggling that flag off will make it consider the alpha premultiplied.

1 Like