I upped the physics solver settings within project settings which seems to have ironed out the object movement and drop when done from server side, but when client side makes a change, it’s not replicated back to server.
And if client tries to change things and server goes to change after then the objects start to scale physically without scaling induced.
super weird.