I’m using the stereo panaromic plugin to render 360 video along the walking path of my character, may i know is it possible to capture the corresponding sound in 3D spatial audio using the latest audioengine?
*ps The 360 will be uploaded to youtube, and youtube supports spatial audio.
Hi fspar30, this is a really interesting idea, I wasn’t aware of this tool until now. I’ll check with the VR team how this tool currently exports audio, but it would be amazing to support Ambisonics Mixdown so that it just works when you upload it onto YouTube or Facebook or something.
Hi Dan,
I think the stereo panaromic tool does not capture sound.
What if we want to capture spatial audio just for a regular sequencer movie?
Audio capture to file is something we like for many, many reasons–from debugging to interesting procedural implementation ideas, so this is definitely on our radar.
Spatial sound has been on youtube for a long time, there are many 360 videos out there with ambisonics. Can’t believe Unreal Engine cannot output this format to file.
As an audio output format, what value does ambisonics bring except as 360 video support? Don’t get me wrong, that’s cool and I hope we can support it some time down the line! But ambisonics isn’t a speaker map, it’s a spherical representation of sound pressure from multiple vectors and necessarily requires a decoder to interpret into a speaker output.
So the architecture of a game audio engine is usually something like 3D sound source interpreted by a speaker map–usually surround–or sent through HRTF. These processes output to speakers, stereo or surround, but ambisonics doesn’t output to speakers–it doesn’t output to anything–so there’s no natural evolution to use ambisonics as an output format.
In other words, normal operation is like this:
3D sound source => Surround/Stereo speaker panning
Or through HRTF
3D sound source => Stereo speaker (intended for headphones)
But with ambisonics, it would be like this:
3D sound source => Encode to Ambisonics => Decode from Ambisonics => Surround/Stereo/Binaural speaker mapping
Ambisonics in the normal use case as an output would act as a middle-man, but not a very good one, because you lose spatial resolution (particularly with First Order because it only supports 3 vectors of audio pressure: up-down, left-right, front-rear).
So it’s not a normal game audio process since we already do 3D audio output better without it because we go directly to the speakers.
Now, don’t get me wrong, Ambisonics as a sound source (like an audio skybox) is powerful, and we’re definitely into getting it working on that front–but as an output file format? I mean, there isn’t any native audio capturing going on, so how would that even be a normal thing to support? I think it’s cool, and I hope we get into it, but I don’t think it’s unbelievable to suggest that we don’t output to file in this format since we already don’t output to file in any audio format.
But yeah man, I think it’d be cool, specifically for 360 video, so I hope this is something we can support in the future! 
Your entire undertone feels like you’re never going to support it.
I don’t worry that Epic doesn’t support it as Unity does. And i am switching over.
Hey J, I’m sorry you feel that way.
I definitely think that Ambisonics file playback is important and in general supporting a wider variety of multichannel files is definitely on our roadmap–including Ambisonics file playback.
However, Ambisonics file encoding for capturing 3D audio into FOA format and exporting a file is only something we’ve just started talking about here (mostly due to this thread). In general, we want to support audio file capture and export both for debugging and QA purposes as well as for its potential in procedural audio applications. Depending on how we end up implementing that, Ambisonics may be included–but it’s definitely a more involved implementation than simply capturing an outgoing audio buffer and copying/committing that to a file, because it’s not a normal output format.
I’m curious if you have an application for Ambisonics file export that is other than for 360 video support because I can’t see its use otherwise. Maybe for simplifying a soundfield? I don’t know.
What are you trying to create?
Hi, @ - has there been any progress made on capture soundfield audio along with stereo pano stills since this post last year?
To answer your question about use cases, Beyond capturing final game audio for marketing etc., the development needs that I see for capturing ambisonic audio, even something as limited as FOA, is as a guide/reference when using the videos for designing sounds offline in a DAW, for quick reviews/approvals, and for capturing FOA audio of ambient beds when it makes more sense to have a FOA stream than to spatialize sound cues, especially for mobile when using a mobile optimized renderer like Google Resonance and balancing CPU vs. streaming costs.
Hi Viktor, we just released Submix Recording in 4.20 which allows you to record the stereo output of any Submix in the graph including the master out. Most of our focus this past year has been to support all of our various platforms and to work toward ensuring the new Unreal Audio Engine is properly battle-tested, optimized, stable, hardened, etc. Our current primary objective is to ensure that the new Unreal Audio Engine is AAA shippable. We’re eager to work toward this goal so we can get out of Early Access state.
Submixes are where spatial audio is encoded to the device format. As an efficiency, we initialize the Submixes to whatever the device output format is. This way people with stereo devices (like an iPhone) aren’t rendering Surround sound (which would cost a great-deal more resources). We are interested in channel agnostic approaches to Submixing but FOA is not as well resolved in the planar vectors as a 7.1 surround, which is our current max surround support format. Yes, FOA gives you a vertical vector, but it is just two directions.
With that said, we do support Google Resonance and FOA soundbeds decoded through either Resonance or Oculus.
Again, as we look to the future, we are interested in clever approaches to soundfield submixing.
Hello! Are there any updates on this? I’m about to export 360 video with the updated Stereoscopic Panoramic plugin and for now I’ll have to add the audio to the mix in Premiere, but it would be nice if there was a way of exporting this in engine, if feasible?
Thanks
Hi @ ,
I have done a lot scouring for information about encoding Ambisonic from Unreal and have come up with nothing other than these post a few others asking the question with no response. As far as it being important to Unreal’s future, I would think with the efforts Unreal is making to make it a tool for video and filmmaking it would be great if there was much better ability in this area. For example, it would be great to be able to have simultaneous listener positions that you can import into a submix that allows you apply mathematical functions like standard math functions, trig and even polar to rectangular and visa versa than output these channels into a finished Ambisonic B 1st order or 2nd order (4 or 8 channel) WAV files for use in another DAW.
I have been experimenting with three different listener positions based upon the camera position and rotation of 0:0:0 degrees for Left/Right, and the 90:0:0 for Front/Back, and 0:90:0 for Up/Down and those three listener rotations to three WAV files in the sequencer. I then sum these three stereo channels and divide by three to create the OMNI channel. I then invert each right channel and sum it with its left channel to create the three figure-eight type outputs for Right/Left, Front/Back, and Up/Down creating the four basic AmbiX 1st order 4-channel outputs. Currently I am using Adobe Premiere, but pretty much any DLE or DAW with at least four channel output can be used.
Why is this important? As many here who are using Unreal to create VR content know that Unreal, especially in 4.26 now offering native spatial audio, have experienced how realistic it sounds inside the VR experience, but lose that experience from an audio perspective in exporting it out using the sequencer. However, in saying that, Unreal has not really done much to push the 360 export either, but luckily others have come up with solutions using the cubemap capture tool and adding a few of my own tricks, and source modification in custom 4.26 build to allow for 16K x 8K 360 video in 12bit HDR with RTX enabled. But sadly audio does not live up to the great video I can export and have to create the spatial sound field in Adobe Premiere with help from a few Ambisonic VSTs. However, I am hoping you can point me in the right direction now that I have been able to capture one listener point .WAV and just repeating the process two more times, if it can be done in one iteration, that would be great! In fact I don’t mind if the process is done like it is now in the sequencer to create a single stereo WAV, but make that four independent WAV files or even better, a grouped four channel WAV with AmbiX metatag.
Here is a link to a test video I created for VR experience titled “Hot Cocoa in VR” using the above technique: https://youtu.be/JtG6Gpiprkg
PS I do believe even if you are not watching these videos in a VR headset and just on your phone, having the video AND audio rotate as you move your phone is far better experience. And frankly not all experiences created with Unreal require user input, or 6DOF as I have found in creating relaxation / calming experiences for my wife’s psychotherapy practice. The one above was sent out during the Christmas holidays and I entered it into the MegaJam but sadly ran into to an upload snag that caused me to miss the deadline by 10 minutes. I was never interested in the prizes, so I am thankful that they still listed in the MegaJam entries for 2020. I any case, I am in no way criticizing the work you guys have made, and understand you have to move forward where most impact will be made with a new feature. In fact those who say they will just move over to Unity don’t realize how many features are created to just die on the vine or are broken in future updates. Unity has it frustrations too. I personally just like the look of the shaders in Unreal, and frankly out of the box, far more control without adding a new feature that may or not work in VR.
Hi Dan,
I have been digging into a build of UE 5 Main from Github and noticed that 7.1 audio seems to work now, but I was hoping you would be using Microsoft’s new Spatial Audio API allowing 23 discrete channels or even a Dolby Atmos encoder. As mentioned in my previous post, I don’t need AmbiX export (even though that would great to go from MOV/WAV muxing in FFMPEG) but having the ability to create a 23 channels to use in a DAW would be fantastic, even something like 5.4.2 would excellent to create my own AmbiX 1st or 3rd order elsewhere.
You asked why I just don’t do the entire audio mix in the DAW and mentioned elsewhere, there are great deal of functions in Unreal eg oclusion and TDI that can’t be duplicated easily without spending a great deal of time that would be so much easier if the channels were there.
If I am missing something in enabling MIcrosoft’s API or more channels, please let me know.
Thanks
I don’t think that’s a feature yet. There’s a lot of fixing up of the Submix Graph and its functionality before we have more varied output types, I believe the work on the foundation for that kind of functionality is underway.
Thank you, anyone I should follow or a UE 5.0 project update like the one on Trillo for 4.27? In any case your response will save me a great deal of time in experimentation.
Also, to anyone else wanting a pseudo solution, I have been able to to just run multi-passes and rotate the camera on its X axis. Not sure what type of pattern is picked up, but I have been able to get fairly narrow cardioid path to simulate x,y,z mic channels to remix in Premiere and Reaper as AmbiX 1st and 3rd order.
An example of this method was used to create a 1st order mix for “Hot Cocoa in VR360” now available on the Meta TV: Hot Cocoa in VR360
Hello, man.
Is there any ptogress in Ambisonic audio export/render ?? I really need that :c
It is very important in 360 cinematic creation!
