Yeah, unfortunately the MIDI stuff we showed at GDC was done with the built-in MIDI plugin (in the controller plugin list) which just supports midi keyboard input. I’m trying to get through some high-priority bugs at the moment so I can see if I can have time to write a midi file importer for UE4 so it’s just natively supported.
To do the music stuff with the subtractive synth, we used a tool that converted a MIDI file to a CSV file, then imported the CSV file as a data table, then wrote a BP sequencer for the CSV file. By “we” I mean Dan. He’s resourceful.
So to try out new audio engine stuff, you’ll need to make sure your project is set to use the audio mixer. Either use “-audiomixer” as the command line param or set it up in your project config’s INI file to use it.
E.g. In …/MyProject/Config/Windows/WindowsEngine.ini
Add these lines:
[Audio]
AudioDeviceModuleName=AudioMixerXAudio2
Also you need to enable seeing audio-mixer-only things in the editor. The editor is in a weird state of some new features which don’t work in the old audio engine and stuff which works only in the new audio engine. So make sure you go and turn on seeing audio mixer stuff in the experimental editor settings.
Currently the only platform really tested is PC though other platforms are in the works. New projects won’t use the new audio mixer by default for 4.16 so you’ll need to opt in. Hopefully by 4.17, it’ll be ready to be on by default for new projects.
Note that the DSP code in the audio mixer module is intentionally independent of U-Object-land. The idea is that they’re building blocks which could be used in any number of plugins, etc. The synthesis plugin is the actual UObject wrappers around the audio mixer DSP stuff that does the work to create BP-bindings (via properties or Ufunctions, etc).
The same workflow could be used for your own DSP creations – write your C++ DSP in classes and files that our outside of the UObject/UE4 stuff, then use the USynthComponent inheritence and source/submix effect stuff as binding/wrappers into UE4-land. In that way you could load your own DLLs (if you’ve written plugins for VST or something else) and then just write the UE4 bindings. There’s loads of examples now of how you can do that yourself. Easiest effect to check out is the WaveShaper one. Writing a new synth using the synth component is a piece of cake now.
I’ve been asked if we’ll ever support VST hosting/importing – at this time, it’s a bit ahead of ourselves, but theoretically possible. Certainly possible to just write BP bindings/wrappers around your code by hand in the new architecture, assuming you’ve written the core of your DSP stuff in such a way that its independent of VST/AU, etc.