Steam integration problem, cannot see any servers.

I am trying to integrate Steam to my game, but I encountered some issues, the client who wants to join never sees the hosted game/server. I decided to download the Multiplayer Shootout example and try to integrate Steam to that, so I can check if the problem is with my project. I followed the steps mentioned here: Online Subsystem Steam

However, I still cannot find any servers, no matter who is the host. I tried it with several friends, and even on my own computer running two instances/accounts of Steam using a virtual machine, no luck.

My question is basically, is the Multiplayer Shootout game suppose to work with Steam after installing the SDK and configuring the DefaultEngine.ini of the game? If yes, what am I possibly missing? If not, is there any aviable tutorials/guides about the proper way of creating a functioning multiplayer game using Steam?


[EDIT] The Steam overlay comes up, so that part is working.