HMD and Vive tracker (3.0) calibration

Hi there, hoping you are all doing well.

So I have a small issue using the vive tracker 3.0 in unreal engine. I am using 2 vive trackers to represent a rotating turret. (I guess it could be considered similar to have used the 2 vive controllers to simplify things !)
My current setup is a tracker on one of the turret feet and another one on the rotating part.
The first tracker is used with position and rotation to say where the turret is in 3d, the other one is used with positions and FindLookAt’s for the orientation.

I am using LiveLinkXR (because all tutorials I found used this for the vive trackers).
There is a Manager that takes devices name and related actors and updates actors accordingly, and also allows calibrating.

My issue comes with the HMD.

I can use LiveLinkXR with the HMD the same way as a tracker (ie. associate an actor and let the manager calibrate / update).
However I can’t specify the player actor since it hasn’t spawned yet.

Thus the manager can’t really update the player actor’s camera AND furthermore, ue automatically associates HMD movement to the camera.

I tried to update the camera position every frame but it gets overriden by ue with HMD movements.
I made a parent component (to the camera) that inverts the camera relative transform and the root component is updated every tick with the HMD-associated-actor’s position.

It does kind of work but tbh it’s extremely uggly and even though I have good specs (2080ti, 32g RAM, SSD, etc…) the vr view feels very jerky.

And know, I wanted to know if anyone had a same situation (tracker - HMD) and how you handled it?
Did you use LiveLinkXR? How did you handle the HMD? Did you use something else ?

Thanks in advance !