I finally got a basic Virtual Production setup running using VIVE trackers. There seemed to be a lot of interest on how to do this so I thought I would start a fresh post on my example project and tutorial videos to make it easier for other people to get started. The base setup for the project works with VIVE headset, vive tracking pucks. The project can be used with almost any webcam or video input device that works with Unreal, right now it is setup for the Aja KONA-HDMI but should not be too hard to modify to use another camera. It could also be modified to work without any tracking gear which would make the setup really cheap, but without a tracker you can’t move the camera while filming.
The project handles delaying the tracker data to sync it up with the video, which is usually several frames behind. It also includes a tracked object (kind of a big cartoon hammer) that follows a second VIVE tracker you can hold in your hand. You could attach the tracker to a handle or selfie stick to make it easier to hold.
It is setup to do background removal using a greenscreen so you can put yourself in the middle of a totally virtual set. There is a tracked garbage matte so no matter how you move the camera, only the real world area where the green screen is will be seen.
The project only uses the “3rd person” starter map so it is really small and quick to download. But it works just as well with a more detailed map. The same setup can also be used to insert virtual characters and objects into the real world, like a CG character sitting on your living room couch.
When you look at my stuff, please remember these are just starter projects, in many cases the lighting is kind of bad and there is no color correction. I’ll be showing how to improve this in future videos.
Below is the whole setup I have, you can use cheaper (or more expensive) stuff than this. You can modify the project for almost any kind of tracker or no tracker if you don’t want the camera to move. It can also be modified to work with nearly any type of camera and video capture card, or even a webcam. So it is possible to get started pretty cheaply.
PC with NVIDIA 2080 TI card (the example uses less than 20% of this)
VIVE PRO vr bundle
2 VIVE tracking pucks
AJA KONA-HDMI 4 input HDMI capture card
Full-frame camera with 35mm lens and HDMI output
Green Screen (I started with a 10 foot wide one from amazon, my current one is bigger)
Happy to answer questions, if you have any suggestions they are welcome, I’ll be doing more videos on this in the future.
Please post your own work too, I want to see it!
Project (UE 4.23) is here GitHub - MiloMindbender/UE4VirtualProduction: An example Unreal Engine Virtual Production Project
Youtube examples and tutorials here: https://www.youtube.com/user/GregCorson