Audio Engine Updates Preview - Feb 2nd - Live from Epic HQ

WHAT

With a brand new version of UE4 on the way, Aaron joins us to talk about some of the upcoming audio engine features. Audiophiles can expect to learn a lot about what is in the pipeline for sound engineering in UE4. Join us this Thursday to hear all about it!

WHEN
Thursday, Feb 2nd @ 2:00PM ET Countdown]

WHERE

WHO

Aaron McLeran - Sr. Audio Programmer - @minuskelvin](https://twitter.com/minuskelvin)

Feel free to ask any questions on the topic in the thread below, and remember, while we try to give attention to all inquiries, it’s not always possible to answer’s questions as they come up. This is especially true for off-topic requests, as it’s rather likely that we don’t have the appropriate person around to answer. Thanks for understanding!

**You can download the audio demo videos here:
**Reverb Preview
Reverb In Game
4.16 Sneak Peak
Archive:

Oooh I’m excited!

Finally :stuck_out_tongue: Current audio system is super hard to control and extend, i hope this one will do better job

Hey excited for the audio engine updates! :slight_smile:
Will there be a way to implemented sound input (microphone) into my ue4 projects with only blueprints?

Woo hoo! Been waiting patiently for this day to come. Can’t wait to hear the details! :smiley:

Hi guys ! Will it be possible now to record sounds in movie built in the sequencer ?

My question is still the same - when can we use UE4’s new audio on Android/Gear VR with Oculus reverb/HRTF spatialization without FMOD? :o

Super curious about spatial audio and gpu traced effects and EAX effects =P

Will we be seeing HRTF for VR without any external SDKS/Plugins needed?

I would also like to know if this means spatial audio for Android finally - in particular with both mobile VR platforms: Daydream and GearVR
And for which UE4 version it’s coming? UE4.6?

Aww yiss!
Waited so long for this!

I’m not much of a sound guy at all myself, but over the years of playing games like Rainbow 6 and watching poorly-made sound engines break the immersion and game play for players, I can greatly respect such an update!

OMG! Been waiting for this for years! :smiley:

We won’t need fancy features - just the basics, like: being able to access the current time of the playhead in BP.
So we’ll be able to sync multiple audio layers. (Rather than playing them ALL at the same time and just fading the volume levels)

Also: A solution for beat-matching would be brilliant.

QUESTION:

Are the new features/enhancements/changes already on the github master? If not, when can we expect them? I’d love to play with them :smiley:

New training stream, simple music game :smiley: so we can learn to take advantage of all these new capabilities! hehehehe.

Thank you for the stream today. I missed the later end of the Q&A so this may have been answered already. I get the feeling the first question is answered by inference but I’m asking anyway.

[QUESTION]: Does the new audio system allow you to ‘get’ from the audio to perhaps determine which audio is playing or more importantly at what point in the audio? Similar to a track I could use to make a media player seek. I may have been doing something more complex than necessary in the past but I remember having to store my own time values so I could create a seek bar that would know both the length of the audio snippet as well as at what point in the time it was playing. I felt that it was not an exact science and I remember it suffering from fractions so if I were to set the audio to play at the same time it was at, unreal engine would have some delay problems or inconsistencies.

Another way to perhaps phrase the same question would be “Will the audio system have functionality at some level compared to a media player?”

[QUESTION] Does the audio system allow for me to bring audio into the project from a designated file directory? I’ve hoped to setup a public sharing folder in a final project that my game player community would be able store and/or drop their personal soundtracks (.wav files) to alter the chosen in-game songs. So a use case would be an in-game MP3 player that would have a set amount of songs but could be easily expanded by a consumer to include their own favorite real-world songs (from real-world musicians) without them having to re-compile and rebuild the project with those resources already in the engine.

I remember exploring non-friendly routes of data streaming and server services to try to get this to function. But to no true or satisfying success.

Thank you for your response and your time.

They’re out in 4.15 previews. If you have the source and are a programmer, the new audio renderer/mixer is in the AudioMixer module. Maybe should have called it AudioRenderer since I think people keep getting confused that this is like a mixing console sort of thing or some specific “mixing” feature. I mean, it technically is but it’s not precisely what it is and not what people mean when they say “mixer”. But most people know what a graphics renderer is, and this is basically the analogue of that for audio.

Anyway, the version in 4.15 is the version that we’re using with Robo Recall (at least as close as I could get it). It’s definitely still an experimental release and a use-at-your-own-risk sort of thing since it’s only used on one of our games and not ultra-vetted by QA, at least at the same level as the old audio engine since the old audio engine is A) used on lots and lots of games and our own internal Paragon/FN, etc, and B) is ultra-old so most of the glaring bugs have been hammered out (or are known) at this point.

I also am still finding and fixing bugs in the new renderer at a relatively high rate (although it’s been about 4 weeks on Robo Recall without any new issues popping up) so I’m not 100% confident in recommending new games use it yet. Definitely check it out and let me know if you find any problems.

We’re working on a host of small GDC demo maps that show various aspects of the new features off. The sneak preview in the stream where we showed realtime synthesis will have a lot of demos showing it off. I’m hoping we’ll turn a lot of these GDC demo maps and stuff into new tutorials or sample projects after GDC. One of my goals after I finally get this out in the world and all the platform backends finished, is to switch to doing more promotion and tutorials. It didn’t make sense to do a lot of that using tech that I was planning to throw away or dramatically change. Also, didn’t have the bandwidth. Hopefully I’ll have more bandwidth in the future to do this sort of thing. I think a lot of people’s frustration with the audio engine in UE4 comes from a mixture of legit issues but also just the lack of good examples and tutorials on how to take advantage of the full power of UE4 with audio.

Doing our own rendering means that the answer to perhaps every “can we do X” question is YES. But, it’s always just a matter of time. My hope was the system I designed was flexible enough that it’ll be very easy for anybody with modest C++ skills to do some cool stuff in their projects.

Specific to your question, I did create a utility you can use on an audio component to get a BP delegate function on audio playback. This is actually available in the old audio engine but isn’t gauranteed to be accurate on all platforms (only PC) since every platform handles getting this information differently. This is a perfect example of why the old audio engine was such a pain – any feature I make that has any dependency on platforms (which this one does), would have to be done again and again for every platform OR just not work the same on every platform (like I did for this). But this feature does work in the new audio renderer which means it will work for all platforms (eventually when those platform backends are written, which again, is planned for 4.16).

However, in a more general sense – if getting a BP delegate callback isn’t enough and you want ultra-sample accurate timing, you’ll be able to make your own “source effect” in the audio engine, which gives you access to any source’s audio stream. Then you can count the samples and do whatever timing you need. The power will be in your hands!

So you can technically do this now if you know C++ and can write some code in your project. There’s nothing which prevents you from creating your own USoundWave from scratch and feeding it arbitrary audio you personally load from disk. I could make some utilities to allow you to do this more easily in the new synthesis plugin I’m writing and I just might do it. But in general this goes against the UE4 way of asset management and causes potential issues with serialization, etc. It’d be a risky feature that might expose us to more tech-support debt than I’d like.

As for MP3 support, we still don’t support MP3 playback though the legal restrictions are ending at the end of this year I believe so maybe we’ll be able to get MP3 support soon.

You can already do this in UE4. I added support to get a delegate callback from an audio component to get the playback progress percent. This is only guaranteed to be accurate currently on PC, other platforms use the “app clock” which obviously is not sample accurate. For the new audio engine, this is implemented so will work on all platforms. As I said in a previous post, if you want to get ultra-accurate, you’ll be able to make a source effect and setup a timer/scheduler, etc.

I have some ideas with how I might bring audio-sample accurate timing to BP but it’s tricky since BP runs on the game thread. I think it’s possible though and with the new audio engine, I’ll be able to do stuff like this with more efficiency since I only need to think about it and solve it once rather than re-approach the problem differently for each platform.

Right out of the gate though, any audio event that is triggered on the same game frame is gauranteed to happen at the same time in the audio thread. That alone has dramatically improved efforts to write procedural music. Robo Recall uses a music stitching system written by our talented Seth Weedin in BP that does plenty of beat-matching, loop splicing, etc. It’s a pretty sweet system and works great in the new audio engine. The day we turned on the new audio engine btw, Seth was extremely excited because of the automatic timing improvements. The shakiness of the old audio engine was such that they were on the verge of cutting his painstakingly authored BP music system but the new audio engine saved it :slight_smile: