Hello,
usualy in Dedicated Server based games there is no window with a character that you can use. On some tutorials there is a window that says “Server” and the other window says “Client”.
When i run 2 players i see “Client 1” and “Client 2”.
How do i even connect to a dedicated server?
It sounds like you are running in dedicated server mode. You can disable that at Play -> Run Dedicated Server.
Dedicated server is windowless instance of the game. You should be able to switch World in editor to check what is going on there.
To connect to the server you should connect by ip, or Create Session on dedicated server and find that session on clients and then join it.