Use StereoPassIndex to render left/right eye not working in UE 5.5 and 5.3

Development Rendering
, ,
1

We want to render textures, shared by other application, in frame sequential stereoscopic 3d

StereoPassIndex

StereoPassIndex is always 1 in UE5.x

In detail:

We successfully used

to modify a material in the material graph based on the left/right eye in a stereo setup in UE 4.27, and found multiple sources that do the same thing. We could not get this to work in UE 5.3 or 5.5, but found no posts or reports that mention any change in the behavior of this property. Other stereo techniques such as different view frusta in ndisplay bassed on left/right eye are working as expected, in multiple stereo approaches (side-by-side, frame-sequential).

This was tested on a PC with a nvidia RTX 6000 with the driver version 581.42 installed.

We hope there is a logic and resolvable reason we couldn’t find yet or somebody can point us to a different solution.

Thank you, Daniel

[Attachment Removed]

2

Steps to Reproduce

  1. set up blank project
  2. copy M_LeftEyeGreen_RightEyeRed from a VR template project, or create a new material that uses a custom node and changes output texture based on this manual: https://www.dariomazzanti.com/blog/stereoscopic-3d-inside-stereoscopic-3d-unreal/
  3. create an ndisplay configuration + actor with at least 1 node. to make sure ndisplay stereo works, use an IPD > 0 and place the ndisplay actor looking at a mesh that will be rendered from different viewpoints based on the IPD. assign the stereo material to the mesh
  4. set up ndisplay/switchboard, and in switchboard settings select any of the stereo rendering modes (frame sequential, side by side or top bottom)
  5. run.
  6. verify that IPD-based stereo works by checking the view of the mesh in the L/R render images
  7. verify that the material does not work as expected: in our setup we always see the material’s “right” output (in both L and R images) - compared to the local in-editor view (mono) where we always see the “left” output

[Attachment Removed]

3

Hello Daniel,

Indeed, this infrastructure has undergone a lot of changes since 4.27, and access to this value is not supported at this moment, so we don’t have a quick fix for the time being. However, we understand the need to expose some rendering specific items to address stereo needs and will be looking into that a bit later, I can’t promise yet but maybe that’s something for 5.8 to look into and that could be backported to older version. I will keep you updated in a few weeks.

thanks for bringing this up

vitalii

[Attachment Removed]

4

Thank you for the insights Vitalii! It is good to know with certainty that is it an issue with the engine and not only us missing something. Could you suggest an alternative solution?

We want to render two different textures alternating (left and right eye). The images are input to Unreal e.g. via spout and should be rendered frame sequential.

[Attachment Removed]

5

Daniel,

To substitute the image in frame sequential stereo output, you would need to intervene in the rendering pipeline.

You can check out FDisplayClusterDeviceBase::RenderTexture_RenderThread code, where all the final copying into the back buffer happens.

[Attachment Removed]

6

Hi Vitalli,

I am taking over for Daniel.

We did some further digging into this and we were able to get it to run in 5.3 and 5.7. My colleague had step-by-step guides generated for both version. Maybe this can help to restore this feature in a future UE release.

I was able to apply those changes to the engine source and build a working test-game with a stereo-material as described in https://www.dariomazzanti.com/blog/stereoscopic\-3d\-inside\-stereoscopic\-3d\-unreal/

UE Stereo Mat Fix Guide

I hope this link works, there should be two .md files in there, for 5.3 and 5.7

best,

Manuel

[Attachment Removed]

7

Thank you, Manuel.

We will take a look if that’s something we can incorporate without breaking functionality

[Attachment Removed]