Solved: I was missing the .ico for the game. It is a 32x32.ico file. Once I uploaded the client image that and published the changes to steam it was able to create sessions. In order to test the server/lobby, I also had to create a key for my other test steam account so it would also have a valid copy.
2 Likes