Physics Handle Listen Server problems

I’m using a physics handle to pick objects up in multiplayer but for some odd reason the Host is always Stronger, Faster and more responsive when it comes to picking up objects, I’ve been trying to get this working for a bit now but any help would be greatly appreciated if you need screenshots just give a heads up.