You can leave physics enabled for each client and the server, and allow the server to replicate movement of the physics objects on its end.
This way the response to the clients is quick, and the server can keep everyone’s physical environment in check.
There can be some slightly noticeable movement artifacts from this from time to time, but its nothing too dramatic from what I’ve seen.
The second approach would be to enable physics of the objects on the server only, and replicate their movement to the clients.
Client actor copies will interact with the physical objects on the server in each of these implementations.
I would test each of these methods to see which one works best for your game.