I’m trying to setup a small virtual production system with a green screen, and I’m having trouble getting a useful result from Unreal Camera Calibration’s nodal offset calculation.
I’m using vive trackers directly connected via LiveLink. I use a checkerboard, and have tried tracking the checkerboard with 1 vive tracker, but the reprojection error is always huge and I can’t figure out how to improve it. Any suggestions on improving nodal offset calculations?
Which version of UE? I would suggest getting on 5.6 if not already as there are a lot of improvements to the solver. Make sure the axis of the checkerboard is offset correctly to match the location and orientation of the Camera Calibration Checkerboard in UE. Large reprojection error suggests the sensor size is set incorrectly. Also double check in the Lens file that the imae resolution and the camera feed info are correct. Example sending a 3:2 image from picture camera inside of a HD feed you get image resolution of 1920x1080 and the camera feed would be 1920x1620 because of the black bars on either side of the 3:2 image. Don’t send any mask or hud with your feed to the lens file.
We moved up to 5.7 and 5.8, but still encountered large (500-800 px) reprojection errors. We have a clean feed coming out of camera, but had some strange cropping happening (black bars on two edges). We’ve also been using a small checkerboard (8.5x11), but will try with a larger board to see if results improved. We will also doublecheck our sensor sizing with this strange crop, and remeasure our tracker offset. Will report back
We’ve managed to get our distortion calibration result below 1 px by using a 2ft x 3ft calibration checkerboard along with a higher camera aperture (f8). Unreal’s nodal offset calibration with a tracked checkerboard still gives us poor results (10-70 px), so we’ve found a manual nodal offset via physical measurements gets us closest.
When we match up our physical/virtual checkerboards, and pan left right, the two quickly don’t match up. It seems like the virtual board moves faster (ie. the virtual camera lens is tighter than it should be). We don’t have any black bars on the composite image. We’re outputting a 2160p feed over SDI from a Blackmagic Pyxis 12K camera, so our effective sensor size is 35.64mm x 20.048mm. Any ideas what could be causing the discrepancy?
What’s your checkerboard looks like for nodal offset calibration? Avoid using symmetric checkerboard, check rows and columns should be one odd number + one even number.
Another gotcha is the measurement from the 2nd tracker to the checkerboard origin. Position offset is easy to get but rotation part is hard to manage.
Our checkerboard is a standard checkerboard generated off of Camera Calibration Pattern Generator – calib.io . Its 15 x 10 squares.
Thanks @Andre_Chen for the suggestion on accounting for rotation in nodal offset. That was what was causing our nodal offset calibration to be off (we didn’t account for rotational offsets). Once we did, the calibrated match up has now become quite close, with just a bit of slipping on the edges of frame between our camera feed and virtual feed.
For our checkerboard tracker, rotation may be causing error there as well. We have noticed that our Vive trackers have a delay and sometimes jump around when out of sight from base stations, so those two factors may be giving us large error in our nodal offset calcs. I’ll test more and report back.