It turned out my problem was a network race condition. I was letting UE4 replicate the attachment/detachment of actors, but also manually attaching/unattaching the same actors in response to replication notifies.
Sometimes the rep notifies would get in first, and when UE4’s internal replication happened, things weren’t in the expected state.
Just removing my manual attachment/detachment on remote clients fixed it for me.