My game is live and getting a 100% audio crash for a small number of users. The game crashes a few seconds after starting up no matter what the user does. (see trace below)
Devices so far: OnePlus One, OnePlus X, ZTE A2017 (ailsa_ii), Moto X Pure Edition (clark), Galaxy S7 (heroltebmc)
The problem is I am not able to repro this locally, but I do have some users willing to help me diagnose via beta builds.
Is there a way to get a log or better crash info via a beta build?
Also, the fact that I personally test on a Galaxy S7, and have never seen this issue, but someone out there does 100% of the time, makes me doubt it is purely device related.
What is the best way to proceed?
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
CM Version: '13.0-20161115-UNOFFICIAL-bacon'
Build fingerprint: 'oneplus/bacon/A0001:6.0.1/MHC19Q/ZNH2KAS1KN:user/release-keys'
Revision: '0'
ABI: 'arm'
pid: 6839, tid: 6936, name: AudioThread 0 >>> com.MyCompany.MyGame <<<
signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
Abort message: 'Invalid address 0x7ff05b00 passed to free: value not allocated'
r0 00000000 r1 00001b18 r2 00000006 r3 00000000
r4 827f8980 r5 00000006 r6 827f8930 r7 0000010c
r8 7dbf9548 r9 00000001 sl 7dbba000 fp 827f855c
ip 00000000 sp 827f8500 lr b6ca0e81 pc b6ca35e4 cpsr 200f0010
backtrace:
#00 pc 0004a5e4 /system/lib/libc.so (tgkill+12)
#01 pc 00047e7d /system/lib/libc.so (pthread_kill+36)
#02 pc 0001bc03 /system/lib/libc.so (raise+10)
#03 pc 000184ad /system/lib/libc.so (__libc_android_abort+36)
#04 pc 0001650c /system/lib/libc.so (abort+4)
#05 pc 0001a015 /system/lib/libc.so (__libc_fatal+24)
#06 pc 0006835b /system/lib/libc.so (ifree+622)
#07 pc 000684c7 /system/lib/libc.so (je_free+250)
#08 pc 0000e343 /system/lib/libutils.so (_ZNK7android7RefBase9decStrongEPKv+54)
#09 pc 00009e41 /system/lib/libwilhelm.so
#10 pc 00016cdd /system/lib/libwilhelm.so
#11 pc 0001c871 /system/lib/libwilhelm.so
#12 pc 03a974b8 /data/app/com.MyCompany.MyGame-2/lib/arm/libUE4.so (_ZN16FSLESSoundSource4StopEv+108)
#13 pc 01d12580 /data/app/com.MyCompany.MyGame-2/lib/arm/libUE4.so (_ZN12FAudioDevice6UpdateEb+2008)
#14 pc 0188d618 /data/app/com.MyCompany.MyGame-2/lib/arm/libUE4.so (_ZN10TGraphTaskI18FFunctionGraphTaskE11ExecuteTaskER6TArrayIP14FBaseGraphTask17FDefaultAllocatorEN13ENamedThreads4TypeE+84)
#15 pc 018cf374 /data/app/com.MyCompany.MyGame-2/lib/arm/libUE4.so (_ZN16FNamedTaskThread23ProcessTasksNamedThreadEib+180)
#16 pc 018cf034 /data/app/com.MyCompany.MyGame-2/lib/arm/libUE4.so (_ZN16FNamedTaskThread21ProcessTasksUntilQuitEi+100)
#17 pc 01d31978 /data/app/com.MyCompany.MyGame-2/lib/arm/libUE4.so (_ZN12FAudioThread3RunEv+84)
#18 pc 0190c2f8 /data/app/com.MyCompany.MyGame-2/lib/arm/libUE4.so (_ZN22FRunnableThreadPThread3RunEv+100)
#19 pc 018cc368 /data/app/com.MyCompany.MyGame-2/lib/arm/libUE4.so (_ZN22FRunnableThreadPThread11_ThreadProcEPv+104)
#20 pc 000478f9 /system/lib/libc.so (_ZL15__pthread_startPv+32)
#21 pc 00018c75 /system/lib/libc.so (__start_thread+8)
Well, you would want to make sure you are packaging in Debug configuration in your Project Settings under Packaging. This way, when the project fails you will get more information in the logs.
I would also get the devices as well as the logs from the users who are experiencing the crash so you can compare and make sure they are all the same one. If you are not experiencing the crash on your end, you will also need to find steps to do so, this way you can potentially fix the issue.
I will try putting up a debug build to beta and get them to crash, send a report and see what I get.
One last important question:
If the build is packaged as debug, and the APK uploaded to beta, will it generate a log for the user when they crash? in …/saved/logs?
Or would they need to do things like use NVPack, etc (which IMO is a bit much to ask of a customer)
It depends if it is the device crashing when loading the project or the project crashing. I am assuming their device stays on, and the project closes at a certain point, which would indicate a crash.
I just tried uploading a debug build, but Google Play rejects a debug APK from being uploaded so that isn’t going to work. I think you misunderstand the issue.
These are customers out there with a live build and I cannot repro locally.
That is the crux of the problem I am having, I know how to debug something I can repro.
Is there any way to get more info in a shipping build than what you see in the crash report above? If not can you help me better understand what is happening in the crash report above?
I did a test via a beta build with the user. By eliminating one node, and having the users test I have determined that the crash is triggered by the PlaySound2D node playing a wav file:
Please let me know what information I can give to help diagnose this further, or if you have a suggestion.
Be sure the .wav file format is supported by their specific android device. If the crash is not occurring on Windows or other devices, that could be the culprit. Below is a link to the Supported Media Formats for Android.
“8- and 16-bit linear PCM (rates up to limit of hardware). Sampling rates for raw PCM recordings at 8000, 16000 and 44100 Hz.”
Yeah, you seem to be within the guidelines of what is acceptable and compatible. I would try changing the sample rate or the codec itself to another format to see if that helps. Is it only happening with this particular audio wave file, or is it when you call the PlaySound2D node anywhere?
It is any wav file, we will do some more tests now that the holiday is over.
One other thought I had:
The way our game is setup the sounds are played from audio managers which are spawned when the level starts. Or, they are in the start of a UMG blueprint that gets spawned.
Could loading an actor/widget and then playing the sound in almost an instant be an issue on some hardware?
Example Code:
FrontendSounds = GetWorld()->SpawnActor<AFrontendSounds>(UMyGameLibrary::GetFrontendSoundsClass());
FrontendSounds->OnMainMenuMusic();
//OnMainMenuMusic is a BP event with an immediate Play Sound 2D node
I could forsee your custom setup being a problem, since not all hardware will be able to load a level at the same rate. If your audio is attempting to play before the level has finished loading into memory and drawing to the screen, that could be a place to look for the cause of the crash. Try putting even a half second delay in between the PlaySound2D node and your execution event to see if that resolves the issue.
Latest test:
Same PlaySound2D node moved to level blueprint instead of using the manager, with a 1s delay. User reports same crashes. Reading the docs, BeginPlay doesn’t start until assets are loaded, correct? So thinking we might be barking up the wrong tree here.
So I really worked at trying to understand the call stack better:
That being said, the other audio crash is at the stop of a non-music sound and just has a FATAL after FREE. So maybe concurrency is another red herring.
Clearly on some environments, there is an engine level memory issue that happens 100% when the engine stops an audio source on android. There are other answer threads about this same issue that go nowhere. At this point I don’t believe it has anything to do with my setup.
Also I don’t want to just start excluding devices as I don’t know 100% if it is hardware or something else about their setup.
Someone at Epic who deeply understands the Android audio tech, and how memory is managed, should look at this further.
In order for us to effectively troubleshoot this issue and to have it entered as a bug, we need a way to reproduce it on our end. This means providing a minimal test case where the crash occurs, and or providing us with steps to reproduce the issue in a new blank project on our end.
Obviously, if it isn’t happening solely a blank project, then the issue is with your setup and how the engine is handling whatever it is you are trying to do with playback of your sounds. This could indeed be a bug with the engine, but we need a way to reproduce it on our end in order to determine that is the case.
I appreciate that Andrew, but I just said that I removed everything and just played the sound in the level BP in BeginPlay, eliminating that it has anything to do with my custom setup.
Do you have any more suggestions or should we just give up?
I mean, it is working on 95% of devices. Obviously the issue has to do with something about those problem devices whether it is hardware or software. What it could be I am stumped on though.
My hope with this thread was that someone with expertise in Android Audio tech would suggest to me what info to gather from my customers having this issue, and then give me suggestions on what the cause could be, to give me a better chance of reproing on my end and getting some real traction on this.
The suggestions I have provided thus far are the extent of my knowledge when troubleshooting audio crashes specific to certain devices. Without a repro, or project in front of me, it is hard to tell how or where to start looking exactly.
Keep in mind, not all games that are published are universally compatible with all of those devices for that Platform, so if your game is working on 95% of the devices tested, that can be used as a metric to determine what those devices share, and what could possibly be causing the crash.
Since we just returned from the holiday break, our Audio Devs and Engineers are extremely busy so getting personal attention from them is unlikely. I can try to ask, but you might have better luck continuing to investigate on your own and reaching out on the Forums for more public support.
