This has nothing to do with casting, casting is never the problem. You do not know how to obtain a valid reference - once you have one, there’s no need to cast… Get Actor of Class will only work if there’s a single instance and it’s fine to use if that’s the case. However, how are you going to deal with the rest of the communication? 90% of programming is about moving data from a → b. If you ever feel forced,to use Get (all) actors of Class to fetch and important BP instance, something has already gone wrong. You mentioned optimising - Get Actor of Class is the very opposite.
Perhaps that’s something to focus on in the foreseeable future. Do consider it:
Good luck!