I figured out a way and wanted to post it in case anyone else needs help with this.
- Download and install Spacewar for the user who will be joining.
- Find, download and install Windows Sysinternals Process Explorer. ( Process Explorer - Windows Sysinternals | Microsoft Docs )
- Run Process Explorer and setup a Path filter for “Spacewar”.
- Re-send invitation from host to the other user.
- While Process Explorer is running on the client, click the Steam join request and let it launch Spacewar (don’t worry about it not launching your game).
- Let Spacewar finish loading, then exit the game.
- Bring up Process Explorer and research the list. Find the first entry where it actually invokes SteamworksExample.exe
- Look at the Properties of the entry. The command line should show, at the end, something like: +connect_lobby 109775243872202570
- Setup a batch file to launch your game, and add that to your command at the end: +connect_lobby 109775243872202570
- Launch your game using that batch file. UE4 will then invoke the Steam join and you will be able to test out your game! (OnSessionUserInviteAccepted and OnJoinSessionCompleted should execute).