Physics Actor disappears sometimes when catching it. VR blueprint multiplayer UE5.4.4

I have a project based off the VR template. I made the template multiplayer. I have physics actors you can throw. I can pick them up and throw them.

Issue 1:

When I throw the actor into the air and catch it sometimes on the client (host/server works fine) the object will not be in the players hand and will be invisible. When I release the actor (onDrop of the grab component) it will reappear where it is on the server. I tried making an update event when you grab the actor it updates the location of the actor but I still have the issue sometimes.

Issue 2
Sometimes the client can’t throw the actor. If the actor is thrown and comes to a complete stop you can pick it up and throw it every time. If the actor is still moving and I pick it up sometimes it will not throw and does the thing where the actor just falls from it’s current location (no added velocity). If I pick up the actor when it won’t apply velocity (where it falls to the ground when released) I can use my other hand to switch the object to my other hand and then I can throw the actor every time.

My physics actor is a little complex. When you throw/drop the actor it gets the velocity of the static mesh and sends that info to the vrpawn and spawns an actor at the same location then applies velocity and destroys the original actor. It seems to work well but these issues need to be fixed.

Video shows both issues. First the issue where you catch it while it’s moving and can’t throw again and then the issue where you catch it and it disappears.

Any ideas would be helpful. Thank you.
Pic of the grab event in my vrpawn where it might be happening. Obviously it could be an issue inside the blueprint for the actor also.