If you move the USE raycast from server to the client and pass to the server the item the player is trying to use. This will sort any issues where physics drops an item differently for the player and the server.
If you move the USE raycast from server to the client and pass to the server the item the player is trying to use. This will sort any issues where physics drops an item differently for the player and the server.