About references

Then why do people say “avoid casting” if in order to use interfaces you need a reference ( even if it was upcasted like “try get pawn owner” in the animation ) ,
So it doesnt matter if you downcast it to the character , it will be loaded in memory anyways