Evolving the Synth Sample Player

Development Audio
1

Hi all,

I’ve been exploring techniques for using Synth Sample Player as a polyphonic sample-based instrument (or ROMpler, if you prefer) while trying to work around the fact that it seems to be quite bare-bones comparead with the Modular Synth and the Granular Synth. I understand completely why it’s a lower dev priority: Aaron and Dan have done amazing work overhauling the audio mixer, implementing these new synths and source effects, and documenting/demonstrating their use; Modular and Granular are by far the most exciting new toys, and sample-based stuff (in terms of longer loops) is very well served by existing Sound Cues.

Indeed, on this project I’ll be using Modular, Granular AND Sound Cues…but I also want to be able to address SSP (or a custom version thereof) in the same way as I address the other two synths.

I’m using arrays of up to 12 SSPs, each containing the same (or a similar) sound wave, and then running a looping counter on every new note from a MIDI file or custom timeline note sequencer. Standard round-robin behaviour as you’d find in Kontakt etc. - each sample gets its tail played out to completion rather than one sample’s tail being constantly interrupted by a playback position reset to 0 on each new note. An array of 12 is necessary for fast melodies; with slower phrases you could get away with as few as 4 or even 2.

Some of the limitations of SSP are purely down to the amount of BP logic required to run it like an instrument - but that’s no big deal. On the other hand, there are some things that’d be very useful which, as it stands, will need to be hacked out of the existing C++ (by our C++ coders - I’m no C++ coder by any means, so I’m prone to making wild/dangerous assumptions about what’s feasible). So as well as asking for advice, I thought I’d document here what we’re trying to do in case it helps anyone else, or helps suggest a direction for ongoing synth component development on the UE side.

First, what I’ll ask our coders for:

  • NoteOn/NoteOff (as per Modular and Granular), ADSR, per-note random pan within two given values, per-note random volume within two given values

  • One-shot sample playback (I can’t seem to stop them from looping - stopping them programmatically would work using GetSampleDuration, but that seems to be broken, and GetCurrentPlaybackProgressPercent seems to be mistakenly returning GetCurrentPlaybackProgressTime)

  • Built-in polyphony/concurrency - either by taking one Sound Wave as it currently does and internally instantiating new ones as required, or by taking an array of soundwaves which may include slight variations (good for humanising conventional acoustic instruments)

  • Some new events, and a couple of other odds and ends that’ll occur to me as we go :slight_smile:

Secondly, the thing I’m asking for advice on: would it be better to modify SSP by shoehorning in elements of Modular and Granular (e.g. the NoteOn/NoteOffs), or to modify Modular or Granular into a sample instrument player by shoehorning in bits of SSP? I guess I’m wondering if there’s any efficiency advantage either way, or if one route seems massively more practicable in terms of coder-hours compared with the other. It might be that, since Granular appears to share a lot of wavetable code with SSP (if I’m understanding the code properly), then it might be a case of limiting some of the grain-related aspects in order to make it behave the way I want.

Sorry if it sounds picky: the new components are awesome and, like I say, we want to basically use ALL OF 'EM. They all have a place: Sound Cues for ambient/wildtrack beds, Modular for big synth washes, Granular for quasi-musical sound effects and (hopefully) SynthSamplePlayer for opening up melodic phrases and percussion to procedural manipulation rather than having to trap them in fixed, rendered WAV loops.

Thoughts, advice, suggestions - all welcome!

2

So after all that, I’m now wondering if using Sound Waves with Create Sound 2D is a better way to go. That technique (instantiating a sound per note) generally behaves more like I want a sample instrument to behave, when used with BP logic to handle pitch/volume/start-time/etc. I can disable looping, manage concurrency so as to avoid sample tail cutoffs, and set notes to auto-destroy. Great! Now trying to figure out how to set a horizontal pan position for each ‘note’…

My only concern is that the tooltip states “This is event is only for cosmetic, non-gameplay actions”.

Does anyone know whether this means it’s restricted in priority, or in any other way, compared with in-game spatialized sounds?

3

I also had the idea to improve the SSP. Although in a different way:
Introducing a Sample Editor.

4

Yes, this would be great! Either as part of the SSP, or (perhaps more usefully) as extra parameters for Sound Wave assets that are respected by everything that uses them.

Ironically, Unreal’s original music system in the late 90s did all this by default! In the sense that it supported Impulse Tracker, FastTracker and Protracker modules (as well as its native UMX, which I think was a variation on Scream Tracker’s S3M). IT and XM mods both support instrument envelopes per sample, including the loop/release system that remains today in Renoise (and Milkytracker, OpenMPT, etc if you’re into that sort of thing). Renoise came from arguru’s Noisetrekker, which in turn was a more MIDI/DSP-friendly iteration of Impulse Tracker and FastTracker.

While FMOD will still replay modules, the only actual support I can find for it in UE4 comes via a plugin on the marketplace that appears to have been written for fun in a weekend, so struggles to play a lot of the modules I’ve thrown at it. I’d love to have the skill to do a good plugin, or to properly leverage the undocumented module stuff that’s still hidden away inside FMOD, but I don’t - and my team’s coders don’t have the time to pursue my nostalgic daydreams :slight_smile:

But yes, the sorts of envelopes and loop-points you highlight would make fantastic additions to UE’s built-in audio system and really open up a seam of efficient, dynamic, realtime and very musical possibilities!