If the server is authoritative over everything, then it becomes simpler because, as you said, then it’s just replicating the movement of the cube. There should be no problem with the server pushing it as long as your character is telling the server to push the cube (applying the force of the player velocity to the cube on the server, or whatever method you have for moving it).
There would be a small delay between the client’s character pushing the cube and it actually moving, but you could hide that delay with an animation on the client end (character puts his shoulder against the cube, etc).