Physics Handle is too loose regardless of settings

You can see a video of it here

Starting with the standard FPS demo level, I’m “holding” objects by attaching them to a physics handle and setting the handle’s location every tick. However whenever I move with the character there’s an initial lag where the object doesn’t move for a while.

It looks purposful to give the object the appearance of weight. Trillion newton damping and stiffness settings have no effect, only changing the wobbling around the edges rather than the amount of displacement lag when you start walking. Increasing interpolation does shorten the distance it lags but not by much. I enabled physics substepping and upped the framerate, but even 120fps only reduces it by about 40% and it has other unwanted effects. The displacement is consistent regardless of mass.

I know in VR you can enable Low Latency Update on the motioncontroller to fix it but I can’t find an equivalent for the standard FPS character.