Object replication in MP

I made a multiplayer project where the server can controll a character and as many clients as you want can spectate him (they get a simple pawn only including a camera). The problem is that objects (using physics) do not replicate their movement: eG I spawn a ball and move it around by walking into it with my character… the ball on the client will behave different from the ball on the server. I let the ball spawn on the server and set his propertys on “replicates” and “replicate movement” but that seems to have no effect. Any ideas how to fix this?