Real-time Onset Detection in Unreal Engine 4!
My algorithm uses Fast-Fourier-Transforms (FFT) to analyse the frequency spectrum of the input audio signal. It detects so called ‘onsets’, somewhat similar to the ‘beat’ you clap or dance to when you hear music.
First part of the video shows a simple kick drum test, second part a fairly complex track:
Orange Graph: Onsets -> x-axis: audio position, y-axis: onset amplitude;
Green Graph: Threshold -> x-axis: audio position, y-axis: threshold amplitude.
The algorithm calculates the spectral difference of the frequency bin spectra from an incoming real signal audio buffer frame, applies Linear Predictions to predict the next, not yet buffered value, to increase onset precision, and uses a weightend threshold to finally determine if there was an onset in the current processed frame.
The computations are fairly expensive for real-time usage but it works pretty much in-sync with the playing audio.
Next step is to use those onsets to drive materials and particle systems!
Music (if interested) was created by myself:
LauchBeats - Anomaly