Audio Engine thread

Anyone here really excited about the possibilities of the new audio engine?

I dont know what is planned but if its powerful like a DAW, and I can control it with blueprints i’ve got some great game ideas that havent been done yet

hey hey, glad there is someone here interested in sound! I have some brilliant ideas for games not yet created with this tech, but my skill lies in audio and graphics (and not the coding part of it, creative side) so hoping I can blueprint some ideas up.

Arghhh… Why is Epic always adding new awesome stuff to each version? Now I really have to think about upgrading my upcoming music based multi genre game :D. I already played with the idea of creating some kind of DAW in the Unreal engine and integrate it. I would be able to do it with the old stuff, but with the new version it is much better to work with :D. I’m excited too.

Absolutely excited, so glad to see audio getting this kind of attention. I politely requested Epic consider adding an audio subforum for these type of discussions, share your thoughts if interested:

So looking forward to playing with this tech in 4.16 and beyond…

Great, added another voice to the sub forum thread.

I only just watched the GDC and am pretty excited but Im not much of a coder unfortunately.

I really want reaktor functionality and unsure how blueprints will handle that, say making a logic gate blueprint running at audio rates…? Maybe this is when I should learn coding Dsp. I’ve got a lot of time on my hands at the mo for this

You are thinking similar things to me, playing certain music gives certain effects, so cool the idea of playing something like an mmo where the music being played would influence the game, so background music choice becomes important

Yeah. I always wanted to create something similar to how our imagination works. When we hear music, we get certain feelings and certain images. I always liked the idea of giving music a visibile shape, which reminds me of an enemy type I am creating aswell based on music. It will form itself after the frequencies and have the strength equal to the frequencies playing. I already have a music based sword, where the blade reacts to music and deforms depending on the strength of individual frequencies :). I hope I can preview it soon and finally announce my serious game attempt called “Sounds of Etheraeon”. But first I have to finish the overdue runtime based terraform feature, which - you guessed it - also reacts to music. Even the trees and plants react to the music :D.

When I’m done with integrating most tech stuff, I will finally go to the combat system and I have fun plans :D.
Cool to see that I’m not the only one trying something new in the veins of music :D.

I have plans and ambitions related to the new audio engine, when it becomes fully available…

I’m hyped for this tbh…

Prior to experimenting with UE4 I was coding some generative music apps as a bit of a hobby in Java using Beads library together with libGDX. I came up with a few ideas then for some cool audio experiences and have been trying to port them to UE4 but so far it’s felt like I’m battling the audio system rather than spending time being creative.

Really happy this is about to change!! Any news on when I can get going with it?

Here’s a little vid of a UE4 audio visual experiment I made a while back…

https://youtube.com/watch?v=ttgGDBq15W0

Same here. I chatted briefly with Epic’s audio guy Aaron Mcleran, who indicated that much (though I’m guessing not all, at least initially) of the new functionality will be available through blueprints. Maybe not quite the same as building a synth from scratch in Reaktor, but we can hope that’s the end goal (after several releases and working out the kinks)?

See https://twitter.com/minuskelvin/status/842796593719857152

Should be largely functional in 4.16 according to the GDC talk, though when that drops is up the air. Judging by the past several releases hitting every 2.5-3 months, and 4.15 released Feb 14th… maybe somewhere in mid-April to end of May? I’m completely speculating here :wink:

In 4.16 we’ll already get first release?!
This is sooner than I though… I have to get rid of some annoying stuff to have time available then; there’s a lot of things I want to do based on the new blueprint nodes that we’ll get.

That’s certainly the impression I get from the GDC talk on Youtube (you can even look in the description of the Youtube video):

… as well as Aaron Mcleran’s comments in the recent Audio Engine Updates twitch stream:

… and forum thread (see the series of comments starting here):

I get the impression that functionality and interfaces will continue to get more robust in later iterations, but the all-new engine (including synthesis) should be there in 4.16.

I’m interested to see how many of the new features are right there when creating standard sound cues, whether you need to install plugins to get things like the new reverb and echo effects. I can understand if Steam Audio has to be a plugin, but I’m hoping the more basic (but still really cool) features are there without having to worry about compatibility issues with third party content every time there’s an engine update. Whilst I’m definitely excited to get to grips with the more advanced tools that look to be coming with 4.16, I can picture waiting on Valve to release new versions of Steam Audio after every iteration of Unreal and that could cause some headaches / holdups.

I’ve just cloned the master branch and can see a bunch of the new audio stuff in there:

ca09c9a72a47e7fd3c41575a49aef716d0a6c259.jpeg

Currently waiting on it to finish building and I’ll give it a little test drive…

It looks like the new stuff is added via plugins, the synthesis stuff can be added via blueprints, the output of which can be sent to a submix and effects chain via the details pane (Sound Cues can also be sent to a submix and effects chain via the details pane) and each sub mix can contain an effects chain.
So the mixer appears to offer proper audio routing.

The effects chain allows you to add effect preset slots where there are a bunch of effects to choose from and the effect parameters can be changed from within them.

Looking at the blueprint side for the effects, I can see that they can be added, removed and bypassed but I haven’t yet found a way to adjust their parameter values in real time. I’m hoping this will be possible though.

The modular synth on the other hand can be routed to a bunch of effects and obviously all parameters can be adjusted in real time.

What really excites me here is the Cpp possibilities…There’s loads of example code and I can’t wait to dive in and write some additional modules.

They’ve done a great job on this it seems…now if I can only get the modular synth to make a sound hahaha…

f657d6457d4901050cc2a4ad7ae7a69fbfd9c33a.jpeg

Capture_02.JPG

1bf9e17698637c7c6c27e085e8dc877f8788f273.jpeg

fb9e17d271ed80df63fe72105499400351e76232.jpeg

This looks really cool. I wonder if you’ll be able to import midi files and control them using internal UE synth?

I didn’t see anything out of the box for importing midi files, however writing a plugin to do this wouldn’t be difficult. In fact a few plugins that do just that already exist.

Yeah, unfortunately the MIDI stuff we showed at GDC was done with the built-in MIDI plugin (in the controller plugin list) which just supports midi keyboard input. I’m trying to get through some high-priority bugs at the moment so I can see if I can have time to write a midi file importer for UE4 so it’s just natively supported.

To do the music stuff with the subtractive synth, we used a tool that converted a MIDI file to a CSV file, then imported the CSV file as a data table, then wrote a BP sequencer for the CSV file. By “we” I mean Dan. He’s resourceful.

So to try out new audio engine stuff, you’ll need to make sure your project is set to use the audio mixer. Either use “-audiomixer” as the command line param or set it up in your project config’s INI file to use it.

E.g. In …/MyProject/Config/Windows/WindowsEngine.ini

Add these lines:

[Audio]
AudioDeviceModuleName=AudioMixerXAudio2

Also you need to enable seeing audio-mixer-only things in the editor. The editor is in a weird state of some new features which don’t work in the old audio engine and stuff which works only in the new audio engine. So make sure you go and turn on seeing audio mixer stuff in the experimental editor settings.

Currently the only platform really tested is PC though other platforms are in the works. New projects won’t use the new audio mixer by default for 4.16 so you’ll need to opt in. Hopefully by 4.17, it’ll be ready to be on by default for new projects.

Note that the DSP code in the audio mixer module is intentionally independent of U-Object-land. The idea is that they’re building blocks which could be used in any number of plugins, etc. The synthesis plugin is the actual UObject wrappers around the audio mixer DSP stuff that does the work to create BP-bindings (via properties or Ufunctions, etc).

The same workflow could be used for your own DSP creations – write your C++ DSP in classes and files that our outside of the UObject/UE4 stuff, then use the USynthComponent inheritence and source/submix effect stuff as binding/wrappers into UE4-land. In that way you could load your own DLLs (if you’ve written plugins for VST or something else) and then just write the UE4 bindings. There’s loads of examples now of how you can do that yourself. Easiest effect to check out is the WaveShaper one. Writing a new synth using the synth component is a piece of cake now.

I’ve been asked if we’ll ever support VST hosting/importing – at this time, it’s a bit ahead of ourselves, but theoretically possible. Certainly possible to just write BP bindings/wrappers around your code by hand in the new architecture, assuming you’ve written the core of your DSP stuff in such a way that its independent of VST/AU, etc.

Very cool. and knowing about the midi controller thing is quite interesting. brings to mind some cool ideas.

Thanks for the information. I was wondering why I wasn’t getting a sound out… I can’t wait to get home and update my WindowsEngine.ini now and to make some sounds.

I recall hearing that there’ll be a sample accurate clock, any ideas on where I can find it? I didn’t find any BP node for this so I assume it’ll only be accessible using C++?

Thanks,

ULLS