The dark outlines look like what happens when the alpha gets masked out twice. Stereo layers are composited by the VR runtime instead of the engine. For it to work correctly, the runtime compositor needs to be told that the alpha is already premultiplied. Most runtimes should support this, but I don’t know if there is any support in the Unreal Stereo layer component for doing that. If there isn’t, it might be necessary to either modify the engine, or composite the video on the application side instead of using a stereo layer.
Hi Rectus,
" the runtime compositor needs to be told that the alpha is already premultiplied."
do yo mean there is some settings we need to change on the Oculus VR plugin?
Not sure how to modify based on your advice.
Could you talk a bit more to details?
It’s something that is done in code where the engine interfaces with the VR runtime.
I looked at the code, and it seems for the Oculus plugin, it actually applies the masking in the engine instead of in the runtime. To fix it, you would need to modify the engine, specifically, the FLayer::UpdateTexture_RenderThread() function in OculusHMD_Layer.cpp, and add a way to set bAlphaPremultiply to false for your layer.
For completions sake, the OpenXR plugin does it on the runtime side by passing the XR_COMPOSITION_LAYER_UNPREMULTIPLIED_ALPHA_BIT flag when creating a quad composition layer. Toggling that flag off will make it consider the alpha premultiplied.
Hi Rectus, we tried bAlphaPremultiply to false, however now multiple frames are remaining displayed , looked like ghosting/onion skin effect, meanwhile the dark edges remains visible.
is there anything that we are missing here?
In that case it might be something doing it earlier in the process. Does it still happen if you output the video to the stereo layer without using a widget in between (if that is even possible)?
