Image difference: Field of View vs. Custom Projection Matrix

Development Rendering
, ,
1

On the heels of [Content removed] I think I have the custom projection matrix worked out such that I get the same view of the world as my original implementation. In my original implementation, I set

CaptureComponent->FOVAngle = HorizontalFOV;

(where HorizontalFOV is calculated from a focal length and image size above), and now I use

CaptureComponent->bUseCustomProjectionMatrix = true;
CaptureComponent->CustomProjectionMatrix = ProjectionMatrix(
		FVector2d(focalLengthX, focalLengthY), 
		FVector2d(opticalCenterX, opticalCenterY), 
		FVector2d(float(ImageSizeX), float(ImageSizeY)),
		CameraParameters.skew);

With this new implementation, the projection matrix defines the frustum (and thereby the field of view) that gets rendered. However, the new implementation also seems to have an impact on the lighting and coloration of the resulting image (attached as image.PNG).

The image on the left uses my original implementation (setting the horizontal field of view), and the one on the right uses the new projection matrix. That the rendered objects are in the same locations indicates the projection is done correctly, but I would not expect the color/shadow differences with this kind of a change. Specifically, in the new projection matrix implementation,

  • The barrier has a subdued red color
  • There’s much more road “speckle”
  • The region under the barrier does not have any shadow

It’s tough for me to determine which of these is “better”, but I tend to think the one on the right is more realistic. Is it expected that using a custom projection matrix would change the coloration and details of the image this much?

For reference, here’s the code to generate that matrix:

FMatrix ProjectionMatrix(const FVector2d &FocalLengthPixels,
             const FVector2d &PrincipalPointPixels,
             const FVector2d &ImageSizePixels, const float Skew) {
 constexpr float nearClipPlane = 10.0f;
 constexpr float farClipPlane = 100000.0f;

 const FVector2d Scale = nearClipPlane * 2.0f * FocalLengthPixels / ImageSizePixels;
 const FVector2d Offset = FVector2d(1.0f, -1.0f) * (2.0f * PrincipalPointPixels / ImageSizePixels - 1.0);
 const float Shear = 2.0f * Skew / ImageSizePixels.X;

return FMatrix(
          FPlane(Scale.X, Shear, Offset.X, 0.0f),
          FPlane(0.0f, Scale.Y, Offset.Y, 0.0f),
          FPlane(0.0f, 0.0f, 0.0f, nearClipPlane),
          FPlane(0.0f, 0.0f, 1.0f, 0.0f));
}

The only way I can imagine this changing the image’s color is if the near and far clip planes somehow tell the rendering pipeline to use alternative settings (perhaps for more or less detail?).

[Attachment Removed]

2

Small update, I did another before/after comparison with extremely sterile conditions to rule out any issues I might have seen from (inadvertently) comparing a built game to the editor or something. The only change to the code is what I mentioned above: I use the FOVAngle setting in the image on the left, and a custom projection matrix instead in the image on the right: [Image Removed]It’s pretty clear that the difference comes down to shadows -- the image on the right just doesn’t have them.

[Attachment Removed]

3

Hi,

it looks like the CaptureSource on the SceneCaptureComponent may not have been set or some show flags aren’t enabled. you can select a capture source with this code snippet:

CaptureComponent->CaptureSource = ESceneCaptureSource::SCS_FinalToneCurveHDR;

The showflags can be found under CaptureComponent->ShowFlags. For shadows please check if ShowFlags.DynamicShadows is set to true. There’s an example of other capture settings in FWorldPartitionMiniMapHelper::CaptureBoundsMiniMapToTexture() inside WorldPartitionMiniMapHelper.cpp.

Please let me know if that helps or if you have any further questions.

Thanks,

Sam

[Attachment Removed]