Two strange camera problems

[anonymous_user_07ba773b]

I’m having two issues with the camera setup in the Vive:

1. There is a lag between actual camera movement and the data that I can get about camera location in Blueprint.

2. I am having objects drawn in relationship to the camera being drawn twice.

Breakdown

Here is a scene that illustrates both problems:

The scene contains a cone mesh parented under a scene object which is sat atop a pillar. The cone is set up to point directly at the camera at all times:

1. The Lag Problem

The cone does not point consistently at the camera in a fixed way, there is a significant lag between my movements and the cone catching up to point at me. This video shows the problem (although it is far more evident and easy to see when wearing the HMD):

HMD Tracking 02

In this video, I am moving suddenly between two positions and then stopping. As I move fast away from a static position, I can see that I leave behind the cone a little bit. Once I stop again, it manages to catch up. Despite what seems to be a relatively simple Blueprint setup (making the cone always point at the camera), there is something strange going on here, and I need to find out how to eliminate it.
**
2. The Double Problem**

Stranger still, this cone is being drawn twice, one pointing at each eye. You cannot see this in the video above, because the viewport only shows one eye, so I have mocked up how this looks when in VR:

I would expect there to be just one cone pointing straight between my eyes, but instead the cone is drawn twice. What’s going on here? Do I need in some way to get hold of both eyes/camera individually and use a location that is midway between the two, rather than attaching something to the main camera object? How does one get hold of each VR/HMD eye camera separately?

I’ve been bashing my head against both of these problems for a few days now, have tried countless different setups to try to figure out what’s going on, and I’m getting nowhere. Unfortunately, both of these things being resolved is pretty essential to my entire project.

Can anyone shed any light on one or both of these issues?

[anonymous_user_07ba773b]

Just wanted to add: I thought this was an issue with the HMD perhaps, but I have just tried testing it with the normal controllable camera outside of VR and I’m having the same problem with the lag. In this video I am simply strafing the camera across to bring the pillar+cone into frame:

As you can see, in the middle part of the shot, the cone is no longer pointing at the camera. Given that every tick, the cone should be oriented to point directly at the camera, why is there ever a frame drawn where that’s not the case?!

I have tried playing around with tick groups, but can’t seem to make any difference. Going a bit mad here…

[anonymous_user_07ba773b]

I’m still trying to figure this out, and not getting anywhere.

I thought maybe there was an issue with getting camera world location in a VR project, because after all there are effectively two cameras in VR rendering (one for each eye) so maybe that was confusing things (and might have explained the weird issue with two cones being drawn). So I tried the same setup as above but using the HMD position (from Input -> HMD -> Get Orientation and Position) as the ‘look at’ position for the cone. Exactly the same issues: (1) the same significant lag, and (2) the cone is being drawn twice pointing in two different directions, rather than just once pointing straight at the HMD.

I’ve noticed a few other posts asking about what seem to be similar things, for example:

Attaching HMD Camera to an animated actor (aircraft cockpit) without lag or jitter (Vive)
Camera lag/stutter - how to fix?
UE4 VR Devs : BEWARE

But there doesn’t seem to be any answers. These issues are completely freezing my development, as my project relies on being able to get an accurate position of either the HMD or the midpoint between the two eyes of the VR cameras. This seems like it should be a relatively simple ask, so what is the problem here?

Would really appreciate if someone (perhaps even one of the UE4 devs) could take the time to explain what’s going on here, and how to get an accurate read of the HMD/camera location. Thanks!

[anonymous_user_07ba773b]

Playing with this all day, still no clue where this lag is coming from and how to eliminate it.

Here is an example which is a bit closer to my actual project:

HMD Portal

Let me explain what you’re seeing here. There is a ring of points (where the pink circle is), and a corresponding ring of points further back whose positions are calculated based on the current camera position. Essentially, these ‘far points’ are always a continuation of the line between the camera and near point. I am using the resulting grid of lines to create a portal which changes the appearance of the material behind the portal. Since the calculations each frame are based on the camera position, this portal effect should always be behind the pink disc.

However, as you can see from the video above (which has been slowed down to 5% speed), there is a delay between the actual position of the camera and the position which is being used to drive the calculations for the portal.
The lines you see being drawn are line traces - the fact that you can see them like this shows that something is going wrong, because they should be drawn straight at the camera. We can see them as vertical lines going up the screen because they are being drawn to a point above the current camera position. Additionally, the little red dots you see are the ‘far points’. If there were being drawn correctly, they would always been behind the pink disc.

I am also printing to screen three things: (1) tick counter, (2) whether the camera is moving or static, and (3) whether the far points are moving or static. Again, if this were working properly, there would never be a discrepancy between the camera and far points being moving/static. However, on tick 786 you can see that the camera has stopped moving while the far points are still moving:

This is because, as can be seen, they are catching up with the position of the camera.

Please can someone, anyone, explain to me why this is the case, and what I can do to fix it?!

Note that I am doing a fair few calculations on tick based on the camera position, but this lag is the same whether I am doing this with one point, or 200.

I’m sorry if I’m sounding like a broken record here, but no-one is responding, and I’m totally stuck until I fix this!

[anonymous_user_07ba773b]

OK, I’ve realised I was being pretty stupid about the double draw thing - that actually makes total sense. If you point something directly between your eyes up close, you see two versions of it exactly as you do in my example. So that part of it I was just being stupid.

This crazy lag on the other hand… help!

[Nick_Whiting]

As a quick note, what you’re seeing is the result of late-update, which is something we use to reduce the latency of the system. Basically, right before rendering, we resample the camera position, update, and then draw the scene from the last possible position you were in. That means that the gameplay code that executed before will be slightly out of date with respect to the camera and motion controllers. Both of those will have updated after you set the cone’s position. You can turn this off, but it’s not recommended. You could add objects to be late updated to the queue in C++, but that’s a fairly involved chain.

Glad you figured out the double draw part though!

[anonymous_user_07ba773b]

Thanks for your response Nick, much appreciated!

I had a feeling this lag was probably the result of something useful and important happening behind the scenes. It sounds complicated to go around it, so I won’t bother right now. I have a couple of other ideas to pursue specific to my case which might be able to at least conceal the problem a little.

Thanks again!

[OneShotGG]

@Nick Whiting;653839

As a quick note, what you’re seeing is the result of late-update, which is something we use to reduce the latency of the system. Basically, right before rendering, we resample the camera position, update, and then draw the scene from the last possible position you were in. That means that the gameplay code that executed before will be slightly out of date with respect to the camera and motion controllers. Both of those will have updated after you set the cone’s position. You can turn this off, but it’s not recommended. You could add objects to be late updated to the queue in C++, but that’s a fairly involved chain.

Glad you figured out the double draw part though!

Is there a reason late update isn’t available in blueprint? As a C# guy that came over from Unity not being able to do stuff in late update without making our blueprint plugin a code plugin is frustrating.

[anonymous_user_07ba773b]

Ok, I changed my mind - I’d like to figure out how to add things to the late update.
[MENTION=2437]Nick Whiting[/MENTION]: how would I begin to figure this out?