Async Sweeps do not detect hits unless the local player camera is looking at them UE4 4.26

I have a server authorative setup where clients connect to.
I have NPCs which spawn async sweeps which are supposed to hit the character player actors.
When the async sweeps happen and the player who is the local player in the server is looking at them, then they detect hits normally, otherwise the sweep happens, but no hit is ever found.
I’ve tried setting all the actors to always replicate, but it doesnt work.
Any help? thanks

It turns out the problem was not multiplayer related or the async sweeps but the animations not updating when the character was not looking, which caused the sweeps to spawn in the wrong place. I fixed it by setting

GetMesh()->VisibilityBasedAnimTickOption = EVisibilityBasedAnimTickOption::AlwaysTickPoseAndRefreshBones;

this probably will cause performance problems if not used carefully