I currently have a real stereo camera. I would like to display the 2D streaming image from the camera’s eyes on a VR device to give it an immersive visual effect. Which feature should I use in UE5.4 to achieve this?
Two questions: What platform are you targeting, and how accurate do you want the video to be?
For acceptable accuracy, you would have to calibrate the cameras, sample the HMD pose at the time each camera frame is captured, and reproject the images from the camera projection to the HMD projection. Getting accurate timestamps for USB cameras is not trivial, and doing it properly requires hardware support and special drivers (just adding a static delay from receiving the frame might work acceptably in some cases).
For better accuracy, you also have to preform stereo reconstruction. Libraries like OpenCV can perform this, but produce a lot of artifacting. (I’ve been working for years trying to get acceptable results from this.)
If you want something simpler (in Windows at least), you could just use the video capture library in Unreal, as in this tutorial. Then modify the material to sample the video texture in screen space, and adjusting the UVs to take half the frame for each eye.
https://www.youtube.com/watch?v=j9-dSlZPOLg