Debugging Audio Difficulties

Development Audio
, ,
1

Sep 23, 2021.Knowledge
Article written by Anna L.

This article is intended for those debugging issues with native Unreal audio in UE4. (For debugging third-party plug-ins, please see the following article )

Getting basic information:

When debugging any Unreal system, it’s always helpful to look at the debug logs, and general performance statistics.

While Warnings and Errors attributed to the LogAudio family of log categories are the easiest to notice, the audio settings section of the log also often contains information useful for debugging. In particular, looking at the settings can often make it easier to sort through issues others have had, or highlight a potential mismatch between the expected settings and the program’s behavior. The audio settings section of the log will look a bit like the following:

[2021.06.04-02.11.26:504][  0]LogAudio: Display: Initializing Audio Device Manager...
[2021.06.04-02.11.26:521][  0]LogAudio: Display: Audio Device Manager Initialized
[2021.06.04-02.11.26:521][  0]LogAudio: Display: Creating Audio Device:                 Id: 1, Scope: Shared, Realtime: True
[2021.06.04-02.11.26:521][  0]LogAudioMixer: Display: Audio Mixer Platform Settings:
[2021.06.04-02.11.26:521][  0]LogAudioMixer: Display: 	Sample Rate: 48000
[2021.06.04-02.11.26:521][  0]LogAudioMixer: Display: 	Callback Buffer Frame Size Requested: 1024
[2021.06.04-02.11.26:521][  0]LogAudioMixer: Display: 	Callback Buffer Frame Size To Use: 1024
[2021.06.04-02.11.26:522][  0]LogAudioMixer: Display: 	Number of buffers to queue:	2
[2021.06.04-02.11.26:522][  0]LogAudioMixer: Display: 	Max Channels (voices): 1084
[2021.06.04-02.11.26:522][  0]LogAudioMixer: Display: 	Number of Async Source Workers:	0
[2021.06.04-02.11.26:522][  0]LogAudio: Display: AudioDevice MaxSources: 1084
[2021.06.04-02.11.26:522][  0]LogAudio: Display: Audio Spatialization Plugin: None (built-in).
[2021.06.04-02.11.26:522][  0]LogAudio: Display: Audio Reverb Plugin: None (built-in).
[2021.06.04-02.11.26:522][  0]LogAudio: Display: Audio Occlusion Plugin: None (built-in).
[2021.06.04-02.11.26:527][  0]LogAudio: Display: Audio Modulation Plugin: DefaultModulationPlugin
[2021.06.04-02.11.26:527][  0]LogAudioMixer: Display: Initializing audio mixer.
[2021.06.04-02.11.26:540][  0]LogAudioMixer: Display: 0: FrontLeft
[2021.06.04-02.11.26:540][  0]LogAudioMixer: Display: 1: FrontRight
[2021.06.04-02.11.26:540][  0]LogAudioMixer: Display: 2: FrontCenter
[2021.06.04-02.11.26:540][  0]LogAudioMixer: Display: 3: LowFrequency
[2021.06.04-02.11.26:540][  0]LogAudioMixer: Display: 4: BackLeft
[2021.06.04-02.11.26:540][  0]LogAudioMixer: Display: 5: BackRight
[2021.06.04-02.11.26:633][  0]LogAudioMixer: Display: Using Audio Device Speakers (Realtek(R) Audio)
[2021.06.04-02.11.26:644][  0]LogAudioMixer: Display: Initializing Sound Submixes...
[2021.06.04-02.11.26:647][  0]LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault'
[2021.06.04-02.11.26:648][  0]LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault'
[2021.06.04-02.11.26:649][  0]LogAudioMixer: Display: Creating Master Submix 'MasterEQSubmixDefault'
[2021.06.04-02.11.26:649][  0]LogAudioMixer: FMixerPlatformXAudio2::StartAudioStream() called

Some particular things worth looking into include:

  • Whether or not the Audio Mixer is enabled. Our older Audio Engine and the newer Audio Mixer are different audio engines, and as such contain different types of bugs. In particular, the older Audio Engine is more likely to have inconsistent behavior between platforms, and some newer functionality, such as Submixes, only work on the Audio Mixer.
  • Whether the Callback Buffer Frame Size Requested matches the Callback Buffer Frame Size that is being used.
  • Whether Max Channels, and Audio Device MaxSources, are acceptably large numbers. It’s worth noting that in this context “Channels” refers to the amount of voice slots that can be used concurrently. If this value is small, or if you have large amounts of sounds playing with the “Play when Silent” virtualization mode, this can lead to voice stealing
  • That the name and channel setup of your output device is what you expect. This is particularly important to verify if you are swapping between devices during gameplay, or if you are having issues on some hardware setups but not others.
  • The specific audio or audio mixer module being used, such as FMixerPlatformXAudio2. These control how audio is compressed and decoded, and are frequently the source of glitches that only occur on some platforms, but not others.

(Note: if you are running PIE, you will likely see this section twice in your log. This is because having multiple Audio Device instances is necessary to preview audio from both the client and server. If you are debugging and would like to only see the debug information for one Audio Device, you can utilize the command SoloAudio to make the currently in view window the only active audio device)

In addition, if you have an idea of which audio subsystem is the source of the issue (see next section), or are seeing an unexpected warning or error that does not provide sufficient context to investigate, it can be useful to change the LogLevel of the relevant category. You can find more info on how to do this here.

In terms of performance, while there is not currently an audio-specific tool within Unreal Insights, it can still provide a large amount of worthwhile information. Hitches in any part of the engine can often lead to audio distortion, so it’s worth verifying if the audio glitch corresponds with an increase in CPU and GPU usage, or a drop in frame rate (especially if you are seeing lots of “Waited . . . ms for audio thread” messages in the Debug Log). Similarly, if you are running your game in a mode that creates the Audio Thread (anything but PIE, unless the settings are manually altered), you can analyze the relative time spent on various Audio Thread level calls.

Narrowing Down the Source of the Issue:

Oftentimes, the better sense you have of where the glitch is occurring, the easier it is to fix. The same methods of narrowing down potential issue sources in other issues apply here, such as attempting to replicate the issue on a clean build, and finding consistent repro conditions. However, in cases where those are insufficient to find the source, it can be useful take the order of operations of audio systems into account:

image
image1319×912 103 KB

The Log Debug commands are often useful in narrowing down where the problem is occurring. For example, the console command au.Debug.Soundwaves 1 (or stat soundwaves, in versions prior to 4.26) can be used to verify if a sound is ever active. As such, this can be used to verify if an issue is happening before the Sound Wave is active (such as in the initial gameplay event, or during concurrency resolution), or afterwards (such as during source effects or spatialization).

Issues that are on the level of the Gameplay Thread or Audio Thread are often easier to debug, while the Audio Render Thread, due to its strict speed requirements, has fewer handles from which to gather debug information. This said, most later stages of audio processing can be toggled on and off via console commands, such as au.DisableAttenuation 1, au.DisableOcclusion 1, au.DisableStreamCaching 1, and au.DisableSourceEffects 1.

For issues that appear to originate in the Submix stage or later, sometimes it can be best to alter the existing settings and see if the problem continues. As an example, it may be worth:

  • Unchecking “Enable Submix Sends” and “Enable Bus Sends” on the affected sound(s)
  • Having any effects removed from the existing Master Submix
  • Disabling external spatialization plugins, such as Oculus Audio or Resonance
  • Targeting a different hardware, if using multiple platforms
  • Altering the Codec you are using to one that is known to work on multiple platforms (see the following)
  • Disabling Audio Volumes within the level
  • Altering the buffer size and/or number of async workers on the affected platform(s), within the project’s Audio Settings
  • Enabling advanced stream caching debugging, if using stream caching

External Audio Analysis:

For some cases, looking at the physical audio output of Unreal can provide hints as to what’s going on. In order to do this, you can use the Submix Record functionality to get a wave file of the output of either the entire game (via Master Submix) or a specific Submix, and placing the resulting wave into an external program such as Audacity. Some examples of hints this can provide:

  • There are evenly spaced areas with sound, separated by gaps where the volume is unexpectedly zero: this is usually a sign that the Audio Render Thread isn’t running fast enough, i.e., there are gaps between each rendered buffer. Check Unreal Insights for possible hitches, enable Stream Caching if it isn’t already, and consider raising the buffer size and number of async workers.
  • Using a spectrogram reveals something that looks like there’s an unexpected low or high pass filter on the object: there may be unexpected processing happening somewhere. Try using “au.Debug.SoundMixes” to verify only the expected amount of Sound Class Mixes are present (their frequency effects stack), use console commands to disable systems like effects/attenuation/occlusion, and disable external spatialization plugins
  • The soundwave has long, flat peaks and/or valleys: your sounds may be getting clamped. Check on your AudioHeadroom setting within the relevant platform’s ini file, use “au.Debug.Sounds 1” to verify that only the expected amount of instances of each sound are present and that the volume multipliers are reasonable, disable Sound Class Mixes/Modulation/Effects, and take a look at the volume of your imported wave assets.
  • The soundwave has some seemingly random, unexpected data in a few locations, and looks different each time the game is run: there may be a memory leak that is corrupting the soundwave data. Try running your program with -memstomp, and taking a particularly close look at any procedural Sound sources you are utilizing.
  • The soundwave sounds normal when played through a tool like Audacity, but not in the executable: there may be a problem happening between the Submix rendering, and communication with the audio hardware. Try disabling hardware acceleration, disabling any spatialization processing that happens directly on your hardware or headphones, and altering settings like the channel setup

In Cases the Issue is Inconsistent or Otherwise Hard To Reproduce:

Bugs like these can be some of the hardest to track down, in no small part because of their inconsistency. The standard methods of debugging apply here, such as running with memstomp, analyzing the stackdump in the case crashes are involved, and setting breakpoints on known code suspects to see if a race condition or altered data might be involved.

For audio specifically, common culprits of inconsistent bugs are garbage collection, device swapping, and stream caching. As such, sometimes glitches are revealed by using the technique of calling garbage collection every frame via the console command gc.ForceCollectGarbageEveryFrame 1. Similarly, sometimes low rate bugs can be found by intentionally triggering device swaps, either through the Blueprint function RequestDeviceSwap, by changing the system default output device while the game is playing, or by connecting and disconnecting output devices. For stream caching, it may

2 Likes