4 years later, there is still no reply 
I was facing and digged into this problem. It seems unreal will try to load all soft reference in map and to invalidate/rename related soft references when deleting/renaming actors.
This behavour can be toggled in Project Settings -> Editor -> Blurprint Project Settings -> Validate Unloaded Soft Actor References.
It speeds up the deleting progress when toggled off, at the risk of breaking soft actor references pointing at deleted actors.
In large maps, it may take minutes to delete an actor, so I decide to turn it off.