Hey there, we recently updated to 4.25 and have started encountering a strange bug where sometimes when closing Play In Editor, the editor will come to a complete freeze, resulting in having to close it through the Task Manager. No crash log comes up when closing it this way. I’ve created a .DMP file of the frozen process and I’m trying to read it in VS2019, but I’m not entirely sure what I’m looking at or looking for.
I am, however, finding some references to an “unsigned int” which keeps popping up in the AudioMixer.dll on the AudioMixerRenderThread
Has anyone else had these strange issues? The game will crash even when playing on a basically empty level, but it’s not every time, but it’s only ever when closing PIE. I’m going to upload a couple of screenshots I think might be related to the issue from the crash dump in VS2019. Thanks in advance!
Ok, digging into this, the fix CL i mentioned on twitter didn’t actually make it to 4.25.1! We lost our embedded audio engine QA so this was missed with general QA (actually they did kick back the JIRA this was under as they saw callstacks in 4.25.1 were still happening in our crash reporter).
We’re debating your callstack here and hoping it’s related to the fix we made for a similar-sounding crash. Fortnite audio devs were hitting a similar crash on PIE shutdown and they were fixed with this CL so hopefully it’ll fix it for you.
If not, we’ll want to get your crash dump and dig in a bit more. Any more information you can provide would be useful too – you say it’s a blank project? Is it always a specific machine (or set of machines) and not the other? Does it depend on the audio devices selected in the OS? Etc.
The 4.25 (mainline) fix CL is now checked in CL 13659105.
Fix should be visible there when it propagates to Github from our p4 server:
That is very comforting to know that we aren’t the only ones to experience this bug. Now we can’t wait for the next update! It’s a real productivity killer having the engine crash on you multiple times in a row haha. It’s not a blank project that we’ve experienced the crash in, but it does occur on different computers with the same project. We’re thinking its something we’ve done that doesn’t like something changed in the engine.
Thanks again for looking into this for us and all your help!
Hi There, I’m also running into the engine freezing on PIE exit inside AudioMixer(same callstack as the second image), this only seems to happen when running multiple clients in PIE. I’ve brought in the CL mentioned but no luck and I’m reaching out to see if that issues is still active on your guys side, might give me some insight into whether it’s something specific to my project setup.
Ah, just to be clear – are you saying you are experiencing this with the fix CL outlined above? The fix hasn’t been published yet, it should make it out in 4.25.2. But if you are still experiencing the issue with the fix we need to get a fix in ASAP. Tomorrow is the 4.25.2 deadline for fixes.
Yes that’s correct, we’ve tracked it down to being a mutex lock contention occuring with FSubmixEffectDynamicsProcessor, one of the audio threads is trying to free up a reference to it while the Mixer is being torn down which is causing it to lock up on GetAudioDevice FScopeLock ScopeLock(&DeviceMapCriticalSection) when it tries to handle the destructor.(which gets locked inside the teardown as well). Will spend more time looking at this tomorrow to find out if this is something we can repro in Vanilla UE4 to have a better example to add to this thread.
We were able to reproduce this in Vanilla UE4(plus the changed mentioned in CL: 13659105. To reproduce this we created a new project starting from the Advanced Vehicle Template(C++)., once this is done we created a submix which is assigned to the Engine_Loop_Cue. Finally created a Limiter effect with default members and assigned it to the Submix Effect chain list. Getting this to repro turned out to be fairly difficult as this is a race condition between the two threads however with a breakpoint I was able to get this to happen very consistently. I would add a breakpoint inside FMixerDevice::TeardownHardware where it UnregisterSoundSubmix and then step through till it exits the for loop and the IsInitialzied if statement, once that’s done I continue the debugg and find myself frozen in editor.
We were also able to verify this in Single Player PIE so the multiple PIE instances did not seem to be part of the issue(probably just more chances for it to happen).
Thanks Ramon for the detailed investigation. I’ll get one of my team members to investigate that mutex. It’s related to an FAudioDeviceManager refactor in 4.25 which is attempting to have a more safe way to reference FAudioDevice handles. We fixed a number of issues like this before 4.25 but it looks like we missed this one.
There is another fix in 4.25.2 related to this thread: CL 13828889.
Curious if that CL might fix this as well.
I’m assuming you’re not using the external submix side-chain feature in the dynamics processor. If not, a short-term workaround (until we come up with a better fix for this) is to check if the ExternalSubmix is valid before trying to get the device handle.
It’s probably better to do that anyway. It doesn’t really address the deadlock directly but it certainly won’t deadlock in this instance if we’re not trying to get the audio device handle.
FSubmixEffectDynamicsProcessor::~FSubmixEffectDynamicsProcessor()
{
FAudioDeviceManagerDelegates::OnAudioDeviceCreated.Remove(DeviceCreatedHandle);
if (ExternalSubmix.IsValid())
{
if (FAudioDeviceManager* DeviceManager = FAudioDeviceManager::Get())
{
FAudioDeviceHandle DeviceHandle = DeviceManager->GetAudioDevice(DeviceId);
if (DeviceHandle.IsValid())
{
Audio::FMixerDevice* MixerDevice = static_cast<Audio::FMixerDevice*>(DeviceHandle.GetAudioDevice());
check(MixerDevice);
MixerDevice->UnregisterSubmixBufferListener(this, ExternalSubmix.Get());
}
}
}
}
Sweet thanks for this, happy to report some success. So the CL mentioned did not remove the lock however the suggestion at the end of the thread did. We are not using the ExternalSubmix side-chain feature so we should be good to go with this and we’ll keep an eye out for when the fix is in so we can account for it in the merge. Thanks again for your help!
Hi, we’re also affected by this problem and it really slows down production as it happens about 20 times a day for some of us. We get the same error as in the first image. We were hoping that a fix would make it into 4.25.2, but we’re still getting the same problem. Can you confirm that this will be fixed in 4.26? And do you know of any workarounds we could use that don’t require to modify Engine code (we’re using a vanilla version).
Thanks for letting us know! We were under the impression we had fixed it with 4.25.2, but it’s possible we missed an edge case. Would you be willing to provide us some more information? For example, do you have a crash dump or call stack?
I also see that you’re using a Vanilla version - did you download the debug symbols along with your install? That will allow you to walk through Visual Studio without compiling from source, which can be helpful in debugging crashes like this.
We’re using debug symbols. When the freeze occurs and I pause Visual Studio and investigate the AudioMixerRenderThread I can see the following callstack (it looks quite similar to the first screenshot the OP posted)
That does look like the same glitch - we’re going to run some more stress tests to see if we can narrow down in which circumstances this is still happening in 4.25.2, and will keep you updated.
In the meantime, would you be willing to tell us a bit more about your submix setup? Its possible some submix settings are making this glitch more or less likely to occur.
Hi, I’ve attached a few sound assets that should give you an idea of our setup. This is an example for a button sound “S_SC_Button_back.uasset”. Inside the sound cue assset there is a submix class linked “SFXMix.uasset”. It has the parent submix class “MasterMix.uasset” and uses a AudioMixerSubmixEffectDynamicsProcessor “DynamicsSFX.uasset” as part of the Submix Effect Chain array. We’re using more than these submix classes, but the general setup is the same.
Please let me know if you have any more questions, I can also supply you with more assets, if needed.link text
