Dedicated server crashes when Matchmaking have some problem on EOS side

Hello.

Our game is designed for dedicated servers(trusted server) with session creation, registration and unregistration players via EOS.

The game is created on UE4.27 with some changes. The game using the OnlineSubsystemEOS plugin and EOSSDK version 1.13.1-16972539.

Dedicated servers (trusted servers) are raised on the Linux platform.

We encountered a problem with the crash of the didated server when a session is created, users register or unregister at the moment when there are Matchmaking have some problem on EOS side.

The crash occurs during the processing of callbacks of functions such as EOS_Sessions_CreateSessionModification, EOS_Sessions_UnregisterPlayers, EOS_Sessions_RegisterPlayers, on the call to TEOSCallback::CallbackImpl, ClientData is filled with garbage at this moment.

A data that is sent to the functions EOS_Sessions_CreateSessionModification, EOS_Sessions_UnregisterPlayers, EOS_Sessions_RegisterPlayers is definitely correct.

We tried to break these functions ourselves, but nothing worked.

We also checked that the data sent to the functions is copied internally, because forcibly filling the data with garbage after calling the functions didn’t give any result.

We have tried to reproduce the problem blocking access to EOS in various ways. Everything works as expected.

The problem only occurs when there is something wrong with Matchmaking on the EOS side. Most often, the problem occurs with EOS_Sessions_UnregisterPlayers.

Stack example:

=== Critical error: === Unhandled Exception: SIGSEGV: invalid attempt to read memory at address 0x0000000000000007
[2025.03.03-21.31.25:180][491]LogCore: Fatal error! 0x0000000002514de7 ProjectServer-Linux-Shipping!TEOSCallback<void (*)(_tagEOS_Sessions_UnregisterPlayersCallbackInfo const*), _tagEOS_Sessions_UnregisterPlayersCallbackInfo>::CallbackImpl(_tagEOS_Sessions_UnregisterPlayersCallbackInfo const*)
[..Private/OnlineSubsystemEOSTypes.h:495]
0x00007fc7eda99284 libEOSSDK-Linux-Shipping.so!UnknownFunction(0x166d283)
0x00007fc7edb1070f libEOSSDK-Linux-Shipping.so!UnknownFunction(0x16e470e)
0x00007fc7ed7b6448 libEOSSDK-Linux-Shipping.so!UnknownFunction(0x138a447)
0x000000000555b479 ProjectServer-Linux-Shipping!FEOSSDKManager::Tick(float)
[..Private/EOSSDKManager.cpp:231]


Example when from UE4 with some of our changes:

// CUSTOM CODE BEGIN Added session name
struct FUnregisterPlayersOptions :
	public TNamedSessionOptions<EOS_Sessions_UnregisterPlayersOptions>
{
	FUnregisterPlayersOptions(const char* InSessionNameAnsi) :
		TNamedSessionOptions<EOS_Sessions_UnregisterPlayersOptions>(InSessionNameAnsi)
	{
	}
};
// END


bool FOnlineSessionEOS::UnregisterPlayers(FName SessionName, const TArray< FUniqueNetIdRef >& Players)
{
	bool bSuccess = true;


	FNamedOnlineSession* Session = GetNamedSession(SessionName);
	if (Session)
	{
		TArray<EOS_ProductUserId> EOSIds;
		bool bUnregisterEOS = !Session->SessionSettings.bUseLobbiesIfAvailable;
		for (int32 PlayerIdx=0; PlayerIdx < Players.Num(); PlayerIdx++)
		{
			const FUniqueNetIdRef& PlayerId = Players[PlayerIdx];


			FUniqueNetIdMatcher PlayerMatch(*PlayerId);
			int32 RegistrantIndex = Session->RegisteredPlayers.IndexOfByPredicate(PlayerMatch);
			if (RegistrantIndex != INDEX_NONE)
			{
				Session->RegisteredPlayers.RemoveAtSwap(RegistrantIndex);
				if (bUnregisterEOS)
				{
					EOSIds.Add(EOSSubsystem->UserManager->GetProductUserId(*PlayerId));
				}


				// update number of open connections
				if (Session->NumOpenPublicConnections < Session->SessionSettings.NumPublicConnections)
				{
					Session->NumOpenPublicConnections++;
				}
				else if (Session->NumOpenPrivateConnections < Session->SessionSettings.NumPrivateConnections)
				{
					Session->NumOpenPrivateConnections++;
				}


				if (Session->SessionSettings.MemberSettings.Contains(PlayerId))
				{
					Session->SessionSettings.MemberSettings.Remove(PlayerId);
				}
			}
			else
			{
				UE_LOG_ONLINE_SESSION(Warning, TEXT("Player %s is not part of session (%s)"), *PlayerId->ToDebugString(), *SessionName.ToString());
			}
		}
		if (bUnregisterEOS && EOSIds.Num() > 0)
		{
			// CUSTOM CODE BEGIN Added session name
			FUnregisterPlayersOptions Options(TCHAR_TO_UTF8(*(SessionName.ToString())));
			Options.ApiVersion = EOS_SESSIONS_UNREGISTERPLAYERS_API_LATEST;
			Options.PlayersToUnregister = EOSIds.GetData();
			Options.PlayersToUnregisterCount = EOSIds.Num();
			//  END


			FUnregisterPlayersCallback* CallbackObj = new FUnregisterPlayersCallback();
			CallbackObj->CallbackLambda = [this, SessionName, UnregisteredPlayers = TArray<FUniqueNetIdRef>(Players)](const EOS_Sessions_UnregisterPlayersCallbackInfo* Data)
			{
				bool bWasSuccessful = Data->ResultCode == EOS_EResult::EOS_Success || Data->ResultCode == EOS_EResult::EOS_NoChange;
				TriggerOnUnregisterPlayersCompleteDelegates(SessionName, UnregisteredPlayers, bWasSuccessful);
			};
			EOS_Sessions_UnregisterPlayers(EOSSubsystem->SessionsHandle, &Options, CallbackObj, CallbackObj->GetCallbackPtr());
			return true;
		}
	}
	else
	{
		UE_LOG_ONLINE_SESSION(Warning, TEXT("No game present to leave for session (%s)"), *SessionName.ToString());
		bSuccess = false;
	}


	EOSSubsystem->ExecuteNextTick([this, SessionName, Players, bSuccess]()
		{
			TriggerOnUnregisterPlayersCompleteDelegates(SessionName, Players, bSuccess);
		});


	return true;
}

Hello, can you confirm you are not accessing the EOS SDK on multiple threads? The SDK is not thread-safe. If you can share a core dump and a full set of logs, that would be very helpful. How frequently are these crashes occurring, and are they happening in a live game?

I can confirm that we don’t use the EOS SDK in other thread only in GameThread.

Unfortunately, we don’t have dumps for this problem. But I attached the logs in a separate private topic:

[Content removed]

This issue never happens when EOS matchmaking is stable. When there is a matchmaking issue, about 20% or more of our servers go down.