SceneCapture2D, tick worth of delay when using HMD?

Development Rendering
,
1

Hi, I have an issue when trying to use scene capture 2d while using hmd. Basically, I’m making a mirror. It seems that there is a tiny delay between what the player sees with his eyes, and what the mirror is showing. When there is no hmd movement, things align correctly, but when hmd moves the mirror’s image seems slightly delayed. The delay used to be worse, but after setting PrimaryActorTick.TickGroup = TG_PostUpdateWork; the delay has been minimized to what seems to be a tick worth of delay.

Is there a way to check if hmd location/rotation is applied to my characters camera before/after I use it to calculate my scene capture offset/rotation. It feels as if somehow the hmd rotation is applied after I already calculated the scene capture position/rotation and hence the tick worth of delay. I attached a video, but its quite hard to see on the screen recording. The first part I’m controlling the movement with a mouse, so the hmd is not moving at all, and thus no delay is apparent. I then tap the vr with my hands and you can see the floor reflection break due to delay (in the middle on the far left of the mirror is most noticeable) . Wearing a hmd, the problem is much more pronounced.

Any advice would be much appreciated!

update: Unchecking Lock to Hmd on the character’s camera fixes the issue, but now I can’t “move around”.

update2: Manually setting the camera’s location based on the hmd location in the player’s tick component makes it work more or less as intended when lock to hmd is unchecked. Still does not feel like a proper solution exactly…

FVector HMDPosition;
FRotator HMDRotation;
UHeadMountedDisplayFunctionLibrary::GetOrientationAndPosition(HMDRotation, HMDPosition);
PlayerCamera->SetRelativeLocation(HMDPosition);

1 Like
2

I had the same problem implementing portals in VR. It really did do the trick, unchecking lock to HMD and setting the camera’s location on the player’s tick component before capturing the scene. You’re my savior, I’ve been searching for a solution for the past few days

3

This is not ideal, and you will soon notice that there is an annoying jitter. Try moving your head really slow and you will notice that the hmd does not sync exactly with your movements and might make you sick.

4

I see it, also noticied that problem. I’m trying to find a solution to that, if I find I will update it here

5

I did some search in the source code, to find what bLockToHmd does, and I think I managed to find the solution.

Firstly we must update the position and rotation of the camera before rendering from the SceneCapture2D.

The update should be something similiar to what bLockToHMD does.

TObjectPtr<UCameraComponent> CameraComponent = GetCameraComponent();
IXRTrackingSystem* XRSystem = GEngine->XRSystem.Get();
auto XRCamera = XRSystem->GetXRCamera();
if (XRCamera->UpdatePlayerCamera(Orientation, Position))
{
	CameraComponent->SetRelativeTransform(FTransform(Orientation, Position));
}

I’m doing this in the TickActor from the Player (Pawn),
So my TickActor looks something like this:

void APlayer::TickActor(float DeltaTime, ELevelTick TickType, FActorTickFunction& ThisTickFunction)
{
	Super::TickActor(DeltaTime, TickType, ThisTickFunction);
	UpdateWithHMD();
	PerformRender();
	
}