A couple more demos up on my channel here https://www.youtube.com/user/GregCorson one shows the “gadget” character standing next to me with much improved lighting. The virtual lights match the real lights in the room and the shadows and hair highlights are pretty close now. It needs a bit more tuning to get the ambient light level right, you can see the shadows are a bit darker on “gadget” than on me because of this. I’m also holding a tracker that is getting replaced by a big cartoon hammer. Tracking is very clean and synced up well, delay is about 6 frames.
This is using an Aja Kona HDMI card, genlock on with a 60hz camera and renderer framerate. This should be workable with a USB capture device or webcam but without genlock the real/virtual sync may not be as exact. This is NOT using any kind of mixed reality plugin. The real and virtual cameras are setup to the same settings, then adjusted slightly to match. Both the real world and cine cameras are set to full frame DSLR 16:9. The lens on the real camera is 35mm, to make it match the Unreal camera had to be set to 36.5, not sure why. I did calibrate the camera/lens but it turned out to have so little distortion that it really doesn’t make any difference. Need to try this with a different camera like a gopro and see how well the function to remove lens distortion works.
The project is here GitHub - MiloMindbender/GadgetTest: Test of virtual sets and tracked cameras but to use it you will have to adjust the video and light settings for your environment. Beware it is over 3gb because the “gadget” assets are very large. Also the first time you load it up it will seem to hang around 45% but it’s really just building all the shaders, give it time.
There is also a sample using the unreal “Virtual Studio” demo project, same camera setup but this one uses the greenscreen.
Also there is a mixed reality sample using the LIV vr software and BeatSaber…no unreal here, but it’s interesting. I didn’t matte out the edges so you can see how small an area you can have and still do this.
By the way, none of these are using any extra lighting, just normal room light.