Correct way to replicate destructible obstacles in multiplayer?

Ok let’s say we have a wall of crates in the level. They are all blueprints that have collision and physics.

They are also destructible in that they handle logic of when they are hit they destroy themselves.

Next with replication tricked and everything two players are in the level and see the blocks being destroyed. So far so good.

Next another player joins and then they don’t see the change in blocks they see the all still blocking a pass way however server knows they are gone so they are phantoms. The new player can walk through them because they are gone on the server side.

How does one ensure this change is passed to a new player? Do they need to be spawned when level loads and not be part of the level itself?

This is like a trapdoor on the floor would end up still visually represented when the door was destroyed when before the player joined.

Thanks.