It seems when I close the listen server, the client has AOnlineBeaconClient::OnFailure() called before AOnlineBeaconClient::NotifyControlMessage is called.
This leads to NetDriver being set to null. Because NotifyControlMessage does not validate the pointer, it results in a hard crash.
I still need to figure out why a listen server causes this to happen.