You can use steam connection for example if you want to connect with your friends from your house over to their house. and there are many tutorials on the internet to guide you on how to achieve it.
As for android you can use google play to achieve that and on ios you can use game center and PSN on play station.
This might seem dumb but im not used to work with networking in unreal engine 4.
Is this only for LAN like only inside a router, or does it work from example my house over to my friends house?
So its okay to use Steam to connect even if my game is not even published on the steam platform?
I guess so , when I used steam all I did was configuring my DefaultEngine.ini and installing steam and register an account , and then I followed the tutorials and packaged the game after that and tested it on two different pc with two different connections and two different steam accounts and it worked fine.