Hey Christian,
Unfortunately, this is an Engine bug and unless you’re building from source, the only other option you have is to wait for an update from Epic.
You can learn more about it here: UE 5.1 Steam Sockets Problem
It seems specifically related to Seamless Travel. Am saying this because in my case - and it would be in yours, too -, disabling seamless travel would make joining sessions work again, though not as expected I know.
What our team is doing is: moving on. We are in the early stages of development and have plenty of other things to do. That said, we’re concentrating on those and will take some action once we truly need to join sessions through Steam; which hopefully may be just in time for 5.2 or 5.1.1.
If you need it now, please, head to the link I mentioned above where they are discussing the workaround I also mentioned. Again: it will require the engine built from the source.
Good luck!