Inaccurate focal length from Lens Calibration
Hello, we are trying to use Lens Calibration process described in https://dev.epicgames.com/documentation/en\-us/unreal\-engine/camera\-lens\-calibration\-overview to get as accurate representation of Lens as possible for Tracking purposes. We have tried the whole process but we are running into an issue where the focal length results are inaccurate and increasingly worsens the closer we are zoomed in.
Our setup is in a limited space which may have an effect on the results, also we are using A4 Sheet of paper for the checkerboard that is glued on flat surface. Do you have any recommendation or is there missing steps in our process? Any suggestions will be greatly appreciated
Steps by Step of the Lens Calibration process we do:
Basic Setup:
1\. Create Cine Camera
2\. Setup Live Link Data tracking of our Sony BRC X\-400 camera (just pan tilt data)
3\. Add Live Link Component to Cine Camera
4\. Set The Live link in the Live Link Component
5\. Setup Media Proxy and Media Profile
6\. Create Lens File
6\. Add Lens Component to Cine Camera
7\. Set Lens File in Lens Component
8\. Add the virtual production CameraCalibrationCheckerboard Actor with the correct Rows/Columns/Square size
Lens file setup:
1\. Check if we receive camera input using the Media proxy and we get data from Live Link
2\. Set CineCamera properties \- sensor dimensions, min and max focal length \- according to our camera
3\. Set the Sensor Dimension in Lens File according to Camera specification in 16:9 aspect ratio
4\. We use Spherical Lens Model based on the Camera
5\. With our camera, zoom from min to max, then focus from min to max \- this is needed for Lens file to update min\-max values for normalization of FIZ data.
Lens Distortion Calibration setup:
1\. set the checkerboard
2\. Check Show Coverage Overlay
3\. Check Fix Image Center \- calibration results for the image center were too inaccurate
4\. into Focal Length Estimate we enter minimal focal length
Lens Distortion:
We Set Specific Focus for example 0.5 and then we do calibration process for whole range of Zooms by 0.1 increment for that focus, after multiple zoom calibrations for that focus, we set another focus and repeat calibrations for multiple zooms… (because of unreal interpolation prioritizing focus over zoom)
For calibration, we always cover whole image, with multiple captures up to 80 images. We tried to capture in varied distances and angles in each calibration. After capturing all images, we click apply calibration, change zoom/focus and repeat.
We even tried, to have checkerboard in static position (for example on the ground) and use pan/tilt of the camera, to capture whole screen. (what’s interesting is that for the same zoom/focus values, with same checkerboard position on the ground, we were able to get different results (we deleted the result and repeated the calibration), by 5mm (which is for lens with focal length range 4.4 - 88mm, quite a lot).)
Our verification method, is comparing output of virtual camera with real camera during zoom. In our virtual studio in unreal, we have precisely placed checkerboard, which corresponds with real checkerboard on the wall. Expected result is, that with overlapping images, should be the size difference same during zooming (size difference could be caused by nodal offset too, but it should be constant).
Anyway, results we are getting are definitely not right. Final focal length curve is in some parts decreasing, which is not possible.