I think it might be worse than just leaving folders behind that will not delete. I think it might be a cause of some editor crashes. The problem must be with dead references to actors that no longer exist in the given map.
Nicolas1966 is correct with his steps on how to reproduce.
Amazing that it is still in ue5.