GetAllActorsOfClass() is a brute-force search so it does not scale well and can be slow. Of course if you do not have a lot of actors spawned it might not be a problem.
I have not seen any profiling markers is that function so it is hard to check how much time is really spend getting all the actors.
Marc