Unreal Engine Livestream - Unreal Audio: Features and Architecture - May 24 - Live from Epic HQ

[Logtrix]

I missed the stream. is it possible to see it somewhere?

[echolevel]

@Jay2645

@Minus_Kelvin Do you have a link to that 2017 GDC example project you mentioned? I saw the GDC video, but I didn’t know that project ever got “officially” released.

Additionally, is there any way to make the Synthesis stuff easier to modify? For example, the Sound Cue has a dedicated editor where you can make adjustments and then hear them “on the fly,” as it were. I know that’d be hard to do for the Synthesizer, but it’s a pain to make an adjustment, open PIE, see how it sounds, close PIE, make another adjustment, open PIE, etc. Some way of at least being able to demo what the sound will sound like in the editor would be really helpful for tweaking and debugging, even if it’s not a full-on graph.

The most efficient means I’ve found of bulk creating/editing synth patches is to set up a MIDI keyboard input system (described elsewhere, in the sticky thread about the new audio engine I think) and also a UMG knob/slider/etc control panel to make adjustments during PIE which persist in your modular synth preset bank. This last bit took me a while to figure out, but you can pass-by-reference in Blueprint to make presets ‘stick’. Eventually I added panel buttons for Create New Preset, Save Current Preset, Clone Preset to New Slot, etc. but the quick and dirty solution if you don’t want to make a UMG panel is to release mouse control from PIE and make adjustments to the preset bank array in a second editor window, while hammering your MIDI keyboard for an updated preview of the sound after every tweak. And if you don’t have a MIDI keyboard, just set up a little looping melody with a Timer hooked up to NoteOn. It’ll get irritating fast, but it’ll do the job

Setting up your own system like this is ultimately more flexible than the kind of built-in preview you describe, since you might - like me - end up with a synth that’s layering the Modular Synth, the Granular Synth and a custom ROMpler-style sample synth simultaneously, using a custom preset structure, and you’d want to preview the whole thing to get stuff balanced properly. And given that the Modular Synth is intended to be extended, or used as an example for people who want to design their own, I can’t see it ever getting what would have to be a highly specific and inflexible in-editor/non-PIE preview.

[S-ed]

@Logtrix Yes. Twitch
Later it should be available on Youtube. Not sure why it’s not there yet…

[Minus_Kelvin (Aaron McLeran)]

@virtualHC

**QUESTION: **An engine built-in cross-platform solution is needed to capture audio from the microphone on mobile (Android and iOS). Is this planned to be included in the near future? I believe this is basic functionality and hopefully it will be considered a priority.

Thanks for the stream! Looking forward to it.

We haven’t implemented mic capture yet on mobile, though I agree it would be a nice feature for sure. It’s not quite considered a priority at the moment unfortunately.

[Minus_Kelvin (Aaron McLeran)]

@psalm702

Question #1

For people who are making rhythm games and real-time instruments, we are prone to deal with latency issues.
In theory, the audio alone can be cut down to 28ms (which is great), but we also have to think about “input latency” (for both software/hardware).

Here’s what can add up to the stack…

Software (version 4.19)

1. Audio Latency: 28ms (60fps + 512/22050 setup, w/ new audio mixer enabled)
2. Input Latency: 33ms (60hz)

Hardware (w/ wide range of variance)

3. Keyboard: 2.83 to 10.88ms (PS/2) ~ 18.77 to 32.75ms (USB)
4. Speaker/Headset: Bluetooth is the worst…

= almost 100ms (PC can be the worst platform)

Are there any future plans (e.g. implementing “sub frame tick” or maybe something even more awesome) that we can look forward to?
Are there any other ways to improve audio response time? Can we really make (playable) real-time instruments someday?

Also, can you give us more options so we can experiment (e.g. removing the lower cap on “callback buffer size”), or would it simply be a bad idea?

Question #2

For people who want to learn DSP (without any knowledge in C++, but familiar playing around with instruments), can you give us an advice on how or where to start? Maybe a learning path? What about PureData (which seems like a visual scripting language)? Also, what do you see in the future?

Thanks!

#1 – for the new audio engine, I’ve implemented a way to automatically schedule audio events from the game thread to the audio render thread sub-frame. However, this only as accurate as the game thread and is not going to resolve your latency problems.

I have a plan on a design to create an “audio scheduler” component which will allow you to schedule audio events at arbitrary rates and times in the audio render thread. Again, won’t solve the issue of input/output latency.

#2 – learning audio DSP is not too bad if you’re good at Math. If you struggle with math (calculus, linear algebra) it’ll be a bit more tricky to pick up. My background is physics so studying audio DSP has been generally more easy going for me. There’s a number of good books to check out to get you started. If you’re new, I recommend starting with a 2-book text Musimathics (http://www.musimathics.com/). There’s a bunch of other good places to go. Pd is cool to start. Miller Puckette’s free book on DSP/electronic music (http://msp.ucsd.edu/techniques/v0.03/book.pdf) in Pd is a good thing to check out but will probably quickly go over your head unless you’re already familiar with the mathematical foundations.

[Minus_Kelvin (Aaron McLeran)]

@Jay2645

@Minus_Kelvin Do you have a link to that 2017 GDC example project you mentioned? I saw the GDC video, but I didn’t know that project ever got “officially” released.

Additionally, is there any way to make the Synthesis stuff easier to modify? For example, the Sound Cue has a dedicated editor where you can make adjustments and then hear them “on the fly,” as it were. I know that’d be hard to do for the Synthesizer, but it’s a pain to make an adjustment, open PIE, see how it sounds, close PIE, make another adjustment, open PIE, etc. Some way of at least being able to demo what the sound will sound like in the editor would be really helpful for tweaking and debugging, even if it’s not a full-on graph.

Unfortunately, it didn’t get released. We intended to dust it off and get it in a presentable mode for people but we have yet to have the time (we’re a super small team spread among many projects here at Epic). We are working on making some sample projects to show people now, we’ll consider dusting off some of the 2017 demos and getting them working on later versions of the engine since it’s a popular request.

[Minus_Kelvin (Aaron McLeran)]

@S-ed

@Minus_Kelvin How Mixes behave when several of them triggered at once/overlapping?

Good question: Different mixes will scale together. Re-application of the same mix won’t double-trigger but will update their timings.

[David_Boura]

Hello! FYI the video is non linked, and did you upload the folders somewhere?

[Jesse.LeBlanc]

Can anyone explain how the baking of FFT analysis works in the new Audio Engine?
So far I’ve got the realtime FFT working. ‘Start Analyzing Output’ of a submix and then using ‘Get Magnitude for Frequencies’ from that submix as pictured in the attached image.
But I can’t use the “Get Cooked FFTData” to return true. What node or process is used to cook the FFTData?
I’ve tried digging and can’t seem to find anything. Even tried turning of ‘Context Sensitive’ for ‘All Possible Actions’

[Jesse.LeBlanc]

@Jesse.LeBlanc

Can anyone explain how the baking of FFT analysis works in the new Audio Engine?
So far I’ve got the realtime FFT working. ‘Start Analyzing Output’ of a submix and then using ‘Get Magnitude for Frequencies’ from that submix as pictured in the attached image.
But I can’t use the “Get Cooked FFTData” to return true. What node or process is used to cook the FFTData?
I’ve tried digging and can’t seem to find anything. Even tried turning of ‘Context Sensitive’ for ‘All Possible Actions’

I figured it out. There’s a checkbox in the actual Sound Wave uasset for generating baked FFTData.