The easiest way to test locally is to load up the editor and click the drop down arrow next to the Play button in the main editor window. There is a box to specify the number of players, put that to 2 then press play and by default it should run the game with the main viewport being the host and a second window will be openned and automatically connected to that host as a client.
One thing to be careful of is that if you are using any of the templates spawn logic there is a bug where players will not spawn if there is a player already occupying the player start location and it will not search for other playerstarts. This means the clients will all start in spectator mode with a first person flying camera. To fix it you will need to write your own spawn point selection logic, or if all you want is to check the connection actually works you can run the game, move the host player away from the player start then open the console on the client by pressing ` or ~ depending on your keyboard setup and type reconnect. Your client should then spawn properly 