Well yeah, you’ll have to handle enveloping fade-outs, etc. If that’s what you want. This is RAW PCM data generation, man. 
You can literally do anything with audio. This is the entry point for real-time synthesis, physical modelling, granulation, literally anything. The only thing left is a similar mechanism for hand-coded DSP/effects processing.