EOS Voice Server Crash and 500/404 Errors After RPC Overflow Client Disconnects

Development Programming & Scripting Multiplayer & Networking
, ,
1

Hello,

In our gameplay, we’re using EOS Voice Chat, which generally works reliably and as expected. As the voice server, we’re using the official EOS SDK Voice Server, with the SampleConstants configured appropriately in the header file.

However, over the past few days, we’ve encountered a recurring issue:

When a client is dropped by the game server due to RPC buffer overflow and then tries to rejoin the voice server, we observe the following:

  • The voice server returns HTTP 500 for the joining client.
  • Shortly after, the voice server crashes at the following line in VoiceSdk.cpp:

Request->MakeRequest(RTCAdminHandle); // ← crash happens here

This occurs inside the FVoiceSdk::Tick() function:

`void FVoiceSdk::Tick()
{
if (!NewRequests.empty())
{
FScopedLock Lock(RequestsMutex);
FDebugLog::Log(L"Processing voice request (%d more queued)", NewRequests.size() - 1);

FVoiceRequestPtr Request = std::move(NewRequests.back());
NewRequests.pop();

// Crash occurs here
Request->MakeRequest(RTCAdminHandle);
ActiveRequests.push_back(std::move(Request));

}

EOS_Platform_Tick(PlatformHandle);
}`

After the crash:

  • New clients attempting to connect to voice receive HTTP 404 for a while for launched server (with same port).

Is this a known issue?

Can you advise on any official workarounds, stabilization suggestions, or best practices to handle this scenario more gracefully?

Thank you,

Harut

2

Hello, can you share logs from the client and server, and a crash dump? This is not a known issue to my knowledge. Are you planning on using the Voice Server sample in production?

3

Hello [mention removed]​,

Our single match lasts 10 minutes, so we initially set

kSessionHeartbeatTimeout = 600.

We have now increased the value to:

const uint32_t FVoiceSession::kSessionHeartbeatTimeout = 2220;

Since making this change, we haven’t observed any crashes so far. I will continue monitoring and will provide a crash dump once issue occurs again.

Additionally, I’ve noticed that the crash most often occurred when a user attempted to join voice chat at approximately the same time the expired session removal logic was triggered.

while (bIsRunning)

{

// update the sdk on the mainthread

EosVoiceSdk\-\>Tick();



// remove any sessions that haven't received a heartbeat within the given timeout

size\_t NumRemoved \= VoiceHost\-\>RemoveExpiredSessions();

if (NumRemoved \> 0\)

{

  FDebugLog::Log(L"Removed %d expired sessions", NumRemoved);

}

// simulate other server activity

std::this\_thread::sleep\_for(std::chrono::milliseconds(30\));

}

As the Voice server was stable during the previous production release, we plan to include it in the upcoming release as well. Do you have any concerns regarding this?

Thanks,

Harut

4

Hi Harut, thank you for these details. I’m speaking with our EOS engineering team. If you get another crash, please let me know and we will look at the dump and logs. Thanks!

5

Hi Harut,

Can you try this change with kSessionHeartbeatTimeout = 600. I think the issue here is a race condition. That lock should be set before we check if NewRequests is empty. I also added a nullptr check as a safeguard.

`void FVoiceSdk::Tick()
{
/** VoiceReqeusts come in from the http api threadpool, but EOS SDK calls must all originate from the same thread.

  • Therefore the VoiceRequests are queued up by FVoiceSdk, which processes them on its main thread, right here as part of Tick.
    */
    FScopedLock Lock(RequestsMutex);

if (!NewRequests.empty())
{
/** Currently just processing 1 request per Tick */
FDebugLog::Log(L"Processing voice request (%d more queued)", NewRequests.size() - 1);

FVoiceRequestPtr Request = std::move(NewRequests.back());
NewRequests.pop();

if (Request)
{
Request->MakeRequest(RTCAdminHandle);
ActiveRequests.push_back(std::move(Request));
}
else
{
FDebugLog::LogWarning(L"New voice request is nullptr");
}
}

EOS_Platform_Tick(PlatformHandle);
}`