UE5.3 AR Actor Location Not Positioning Correctly (IOS)

Development Programming & Scripting
, , , ,
20

Hello,

I did reproduce the issue where objects just weren’t staying fixed to the real world well, the following is a work-in-progress c++ fix that one could apply locally to 5.3 or 5.4. The problem is that the projection matrix FOVs were incorrect. They would have been correct on an older 16:9 iphone in landscape, but way off in portrait and somewhat off in landscape on a recent iphone which has a 19.5:9 aspect ratio, because we would always end up using a 16:9 aspect ratio from the Camer aComponet (unless one modified the camera componet aspect ratio in blueprint).

This fix has the ARKit build the projection matrix itself (like arcore or vr stereo rendering) rather than relying on the fov/aspect ratio/letterboxing system used for non-ar/vr applications. For AR we need the projection matrix to match the passthrough camera image.

AppleARKitCamera.h
Add:
	void GetViewProjectionMatrix(EDeviceScreenOrientation DeviceOrientation, FSceneViewProjectionData& InOutProjectionData) const;


AppleARKitCamera.cpp
Add:

void FAppleARKitCamera::GetViewProjectionMatrix(EDeviceScreenOrientation DeviceOrientation, FSceneViewProjectionData& InOutProjectionData) const
{
	// Use the global viewport size as the screen size
	FVector2D ViewportSize;
	GEngine->GameViewport->GetViewportSize( ViewportSize );
	
	float FOV = 0.0f;
	if (DeviceOrientation == EDeviceScreenOrientation::Portrait || DeviceOrientation == EDeviceScreenOrientation::PortraitUpsideDown)
	{
		// Portrait
		FOV = GetVerticalFieldOfViewForScreen(EAppleARKitBackgroundFitMode::Fill);
	}
	else
	{
		// Landscape
		FOV = GetHorizontalFieldOfViewForScreen(EAppleARKitBackgroundFitMode::Fill);
	}
	
	InOutProjectionData.ProjectionMatrix = FReversedZPerspectiveMatrix(FMath::DegreesToRadians(FOV) * 0.5f, ViewportSize.X, ViewportSize.Y, 10.0f);
}

AppleARKitSystem.cpp
Update the following function:

	virtual void SetupViewProjectionMatrix(FSceneViewProjectionData& InOutProjectionData) override
	{
		if (ARKitSystem.GameThreadFrame.IsValid()) 
		{
			ARKitSystem.GameThreadFrame->Camera.GetViewProjectionMatrix(ARKitSystem.DeviceOrientation, InOutProjectionData);
		}
	}
4 Likes