I’ve been trying to test ambisonics file decoding to binaural (or simple stereo) in Unreal Engine without any sucess .
For the test I have created an ambisonics file in the ambix format with a sequence of noise pulses (which are easy to localize) panned to the left direction. If I decode that file with any VST plugin for binaural or simpler stereo decoding and the rotate the ambisonics track using IEM’s SceneRotator I can very clearly detect the direction of the sound and in particular when the sound is on my left, or right, that direction is extremely clear.
Import the the file named noise_left_ambix.wav to Unreal Engine. Loop it in the SoundWave settings.
Add the file to the scene, do not activate “override attenuation”. The sound appears to come very slightly to the left of center (5º- 10º ). The sound does not change position when rotating the camera.
Add the file to the scene, do activate “override attenuation”. Keep “panning” selected. The sound now changes somewhat when the camera rotates, but I can only detect it changing something like 5-10º to the left or right. At no point when rotaing the camera does the sound appear to come clearly just from the left or just from the right (when comparing with the binaural or stereo decoding in reaper where this is very clear; in know that it is possible that one channel will never get silence).
Add the file to the scene, do activate “override attenuation”. Change to “binaural”. Sounds like the same as in 2).
Set spatialization plugins as follows in the windows platform in the project settings: Spatialization Plugin: Resonance audio; Reverb Plugin: Resonance Audio; Occlusion Plugin: Built-in Occlusion. Change to “binaural” in the sound. No sound output.
Set spatialization plugins as follows in the windows platform in the project settings: Spatialization Plugin: Oculus audio; Reverb Plugin: Oculus Audio; Occlusion Plugin: Built-in Occlusion. Change to “binaural” in the sound. Get a sense the direction is somewhat changing but can’t tell what is the direction of the sound.
Set spatialization plugins as follows in the windows platform in the project settings: Spatialization Plugin: SteamAudio; Reverb Plugin: SteamAudio; Occlusion Plugin: SteamAudio. Change to “binaural” in the sound. The sound now changes somewhat when the camera rotates, but I can only detect it changing something like 5-10º to the left or right. Again, not a correct decode like with the dearVR binaural decoder.
So my question is, I’m I doing something wrong or is ambisonics decoding to stereo or binaural in Unreal Engine not working correctly ?
On UE 5.0.2 I get a result similar to what happens with 4.27.2 but I have an additional problem. There seems to be a copy of the sound (so the original sound plus a delay) playing simultaneously with slightly different location. This does not happen in 4.27.2.
Ok, I’ve finally manageed to find something that works.
So, the strange delay on UE5 is due to the audio compression, switching to PCM or ADPCM the delay disappears. The nature of the sound (noise pulses) makes the problem apparent. The compression algorithm must have changed for UE4.
I’ve managed to get the correct decoding to binaural by :
Setting the Spatialization and Reverb plugins to “Resonance” in the project settings, Windows Platform.
Creating a Sounfield Submix asset and setting it to “Resonance Binaural Spatialization”.
Activating “override attenuation” and setting it to “binaural”.
On the SoundWave details setting the base submix to the created asset.
This works on both UE 4.27.2 and 5.0.2.
The documentation Native Soundfield Ambisonics Rendering states that " After your ambisonic file is imported, you can treat it like any other Sound Wave in the engine, with the important exception that rotation matters. When you play back an ambisonic Sound Wave via an audio component in the world, the engine will rotate the ambisonics bed based on the rotation between the current rotation of the audio component that is playing and the current rotation of the player in the game world." From my experience this not true at all, if you treat the file like a normal audio file the ambisonics stream will not automatically decode correctly to the current number of channels being used (stereo, 4.0, 5.1, etc). I believe the documentation is not accurate. Unless I’m missing something, ambisonics playback only works if the Resonance plugin is used (generating a binaural output), which is not what is claimed in the documentation.
I tried the same thing with the oculus plugin which did not output any sound on UE5.
Thanks for this update - i have also had issues with decoding ambisonics.
By selecting ‘binaural’ you are telling UE to decode the ambix using the Resonance plugin. this will output fine if you are building a project for headphones or stereo, but i believe will cause issues if you want to playback using a multichannel setup (5.1 etc)
Selecting ‘Panning’ it will use Unreals native decoder which i believe works well with FOA.
Having spatialised=true will world lock the rotation, and having specialisation=false will have listener orientation.
You should be able to route the SW directly to a normal submix and pretty much leave it default settings, as long as you use ‘Panning’ - Dont use any of the specific ambisonic submixes though - it has been confirmed to me that they are not working as intended.
It looks like you have solved your issue, but i wanted to also share what i’ve discovered in case it also helped.
In the case of situation 2) above, which is using Unreal’s native decoder, what I’m hearing does not sound correct at all. I don’t get a distinct direction for the sound. For instance if I compare with decoding with blueripple’s O3A decoder - stereo the later is also not binaural but I’m able to clearly hear when the sound is on the right or left.
Also, I wonder if Unreal Engine is capable of doing a correct decoding of ambisonics when outputting 5.1 or 7.1 (I can test that in the future).
Interesting, i’ll have a go over the next couple of days to see if i get similar results, it may be something we need to bug.
As far as i’m aware, if you use native decoding then the upmix to 5.1/7.1 will work well. I will finally be back in the studio soon so will be able to test this better!
I’ll post back here when once i’ve tested to see if we have similar issues.
oh, and i do love Blue Ripple, i used the plugins for several years, there aren’t many that come close
.