Does anyone know why reality capture puts black edges on sides, top or bottom of undistorted images? The image sequence is derived from a drone movie clip. Is there a way to avoid that with full frame borders kept intact so that exported stmaps can be used to redistort plates used to create the model?
Also, is there a way to enforce fixed focal length on solves? Exif data has been stripped in processing but would like to input lens data manually.
One last question for this post. Is there a way to ensure that reality capture does not skip input frames in an image sequence and then export the corresponding camera for that frame as part of the registration export? Or even better, export all of the frame position solves as a single camera?
about your first question, this is quite common behavior, as original image was distorted and the undistorted image has some empty part which are filled by black. You should be able to choose a wanted color in the settings:
About your second question, this can be set in 1Ds view, when you select all images, go to Selected inputs/Prior calibration and set Fixed and wanted length there:
The last one, if the images are in sequence and not unfocused/blurred, then they should be aligned without skipping. Then it is possible to export the cameras with a model as FBX or ABC format.
The black bars for the undistort are a point of concern because most software that we use does not create these black bars. They are unsightly for using as an undistorted plate in a content creation software like Maya or Houdini. PFTrack, 3D Equalizer, Nuke, etc., do not create these black bars as far as I know. If there is a way to undistort while maintaining full frame, that is preferable. Is this possible?
Next there is a concern that STmaps won’t fix this and bring us back to the initial distortion of the full plate because of these black bars. We will test, but if you know the outcome from your own testing, please share it.
You might ask why we would want to use undistorted plates from RC and that is why I asked the third question above.
If we can export all cameras and all undistorted frames regardless of whether they are blurry, we can use Reality Capture as a tracking software solution that also creates a detailed model of the terrain and maintains good frame to frame accuracy in a camera track.
If it’s possible to combine these individual cameras back into one shot camera it would streamline a process that we currently do post-export with a script in Houdini.
The preference is that RC can maintain its present behaviour and cull images that are blurry during reconstruction but also maintain camera position of those blurry images and record frame positions for all images, not just the sharp ones it uses for reconstruction. Then to be able to keep all frames as part of the registration export.
This is from application Help about Undistorted images:
The process of creating undistorted images for export is as follows:
In the first step, the application checks the Image cut-out option. With this parameter, you define which fraction of the image is considered for the undistortion.
1.0 means full image
0.5 means 50% of the image
0 means nothing is visible
NOTE: For fish-eye lenses, we recommend using factor 0.8.
2. The Fit parameter, which defines the section of the undistorted image considered for output, is applied. You can choose among several options so that the image coverage is reasonable.
Outer boundary, Inner region and In between See the scheme below.
Keep intrinsics preserves the camera calibration parameters.
3. In this step the Resolution parameter, which specifies the resolution of the image from the step 2, is applied.
Fit keeps the resolution of the image created in the previous step.
Preserve The resolution of the undistorted image is the same as the resolution of the original image.
Custom You can define an arbitrary resolution.
NOTE: This might not be the final resolution of the exported image. The final resolution is dependent also on the parameters Downscale and Max count of pixels (see the next steps).
4. You can control the final resolution of the exported image by the Downscale factor. Factor 1 means that the resolution calculated in the previous step is used. Otherwise, you can enter a custom number (integer) by which each side of the image is downscaled.
5. In the final step, the parameter Max count of pixels is applied. Use 0 for no limit. In such case the resolution calculated in the previous steps is the final resolution of the exported image.
Alternatively, you can define the maximal resolution of the image in pixels. In such case, the aspect ratio of the image size calculated in the previous steps is preserved but the image is resampled in order to fit the desired number of pixels.
6. Using the parameter Undistort principal point, you can control the position of the center of the undistorted image. Use 1 to shift the optical center of the image into the actual center of the exported image. Use 0 for no shift (see the image below).
|
Undistort principal point = 0
|
Undistort principal point = 1
|
I think it is not possible to export cameras when they aren’t aligned. There is no its position in space, so it could be difficult.
This is very helpful and I appreciate the thorough explanation. I’ve been using RC sporadically for a few years and I haven’t been able to find the help file installed and thought that because of the unique interface it might all be online. Just now I looked again and I do see the help entry under the application section. Thanks for this clarification, it’s great to know that I can go through and find a summary of each tool and workflow options.
I am unclear on whether the different image cut-out settings used can avoid the appearance of the black bars and leave the edges of the frame intact. If so, that’s great news for us.
Regarding the camera alignment of blurry frames, is it not possible to interpolate blurry frames and keep them aligned based on the previous and next frame after? In a drone flight path, there would be over eighty percent overlap of frames so I assume that despite motion blur those frames could be aligned if it was included as an option. Preferably just for the full path of the camera frame by frame and not for reconstruction of the model from sharp frames.
The help file can be also find, when you click and push over any 2D tab:
Problem with blurry images can be, that there is not enough features to connect it with another images. Sometimes it happens, that also blurred images are aligned, but it is not good for model creation. I think it is theoretically possible but it is not a feature in the application. I can create a feature request for this and then our developers will decide, if they implement it.
I’ve found that the undistortion settings in the export registration dialog ‘inner region’ and ‘in-between’ both keep the edges of the frame intact which is what we want. It’s great that’s an option. Thank you for pointing me in the right direction.
Regarding the alignment of blurry frames, it’s great if that’s possible. This drone shot doesn’t have any really blurry frames so they could all be aligned and it would make for a must easier camera export for tracking purposes
There is currently a unexpected output in that image sequences exported don’t have frame numbers at the end by default. Which means they do not load properly in content creation packages for playback. Am I missing a basic setting that enables this as an option? Makes sense that this is turned on by default.
I appreciate you mentioning this to the developers! Is it possible for you to provide an email so that I can explain more in depth why this would be a good added feature? I talked to someone I know at Epic in virtual production and they are interested in the workflow for the same reasons of expediting virtual production workflows.
Regarding the naming, when I choose this export image settings naming convention of 00001… under Export Registration, it numbers the last frame first which I think is a bug.
This also happens with naming convention option file_0001… and I thought it might be because of the order in which I’ve selected the images. Last frames of the sequence are at the top of the input panel so I’ve selected them first. However, once I select the first image in the sequence first and the last one last, the output is still reversed which is not desirable. Is there another setting that overrides this behaviour?