I would like to create a procedural audio component that generates sound from some source wav clips including modifications that come from an algorithm. The USoundWaveStreaming class looks like it may support what I’m trying to do, but I’m not sure how to use it. I created a custom class derived from USoundWaveStreaming, but do not see it available in a Sound Cue blueprint. So it seems I need a “home” for an instance of my USoundWaveStreaming object. Ultimately I’d like to have a USoundNode that I can see and connect in a Sound Cue blueprint that will generate and modify an audio buffer using C++ code and some source audio snippets from a wav file.
I’m new to UE so if there’s a more direct way to accomplish this please point me in the right direction. Thanks in advance for the help/advice!
Im very new to UE too but i think that all classes to be used in a level have to be actor as a parent in some way or the other. And since your sound class is a component you have to attach it to some class for making use of it.
But i dont know if you have to place a sound in the scene to get it played. So there might be a better way for you.
@ Spaehling : Thanks but it feels like creating an Actor is not what I’m looking for. I want a way for my custom object to interact with the rest of the UE4 audio nodes in a Blueprint, so I’m trying to find the most natural approach. It feels like I should be creating a custom Sound Node (deriving from USoundNode class) or a custom Sound Wave (deriving from USoundWave) or both. But I want to avoid re-inventing the wheel as much as possible.
@ Triplelexx : I have used FMOD in the past but I’m looking for a solution using UE4 without FMOD.
@tegleg - Thanks, I’ve scanned some of your discussions here tegleg and sounds like we are trying to do a similar thing. There are several possible ways to set this up. I think I’m getting close to a nice implementation. I will share my result here if successful.
I have a working procedural sound example. But there is a problem with it. I created this by looking at the source code for the USoundWave, USoundNodeWavPlayer, and the USoundWaveStreaming objects. I created two classes USoundNodeProceduralTest and USoundWaveProceduralTest which derive from USoundNode and USoundWave respectively. The USoundNodeProceduralTest is a node that is available in a Sound Cue and acts as a source for the procedurally generated audio signal. When this node get’s parsed it creates a USoundWaveProceduralTest object (only creates the first time) and then calls it’s Parse method. The USoundWaveProceduralTest object is a procedural audio source following the implementation of USoundWaveStreaming (implements GeneratePCMData , bProcedural = true, etc.) .
This all seems to be working fine and plays the pure tone I create and fill into the PCM buffer, EXCEPT, I noticed a “clicking sound” at a very regular time interval. I traced this back to the size of the PCM buffer. This size is set in AudioDecompress.h :
#define MONO_PCM_BUFFER_SAMPLES 8192
The “clicking sound” seems to be coming from a delay whenever the buffer is drained and the GeneratePCMData method is called to re-fill the buffer. If I set this to a much higher value I still hear the click but of course at much larger intervals. (8192 corresponds to 186ms for 44100 sample rate).
I think I’m implementing this system the way UE4 would expect. Is there a way to get rid of this delay (e.g. it should ultimately be fetching the data sooner and double buffering somewhere behind the scenes) ?
EDIT:: I didnt check or used this software and just picked it out of google for reference here what kind is needed!
I dont remeber what i used maybe it was this maybe something else
Anyway, check if your pc is capable of streaming realtime audio.
And if you have there an issue try first energy settings .
usb is standard on some snooze level that turn off.
And depending on your motherboard there are some bios settings aswell. But i didnt needed them so i hope u neither.
Worst case would be that your pc / audio card isnt capable of doing it.
There is a double buffer system. You can see it initialized in FXAudio2SoundSource::SubmitPCMRTBuffers (or FCoreAudioSoundSource) and then it gets replenished in HandleRealTimeSource as needed. If you’re getting a consistent click regardless of changing the size of the buffer I would assume that you have a discontinuity in the data you’re putting in to the array which results in the click.
Here is the source code for the 2 classes. If you add these to a project you should be able to then create the SoundNodeProceduralTest inside a Sound Cue and connect it’s output wire to the speaker, hit play cue, and here the tone that is created (plus the “click” that I’m trying to debug).
// File: SoundNodeProceduralTest.h
#pragma once
#include "Sound/SoundNode.h"
#include "SoundWaveProceduralTest.h"
#include "SoundNodeProceduralTest.generated.h"
UCLASS()
class AUDIOTESTPROJECT_API USoundNodeProceduralTest : public USoundNode
{
GENERATED_UCLASS_BODY()
// Volume of the sound [0-1]
UPROPERTY(EditAnywhere, Category="Sound Source Properties")
float Volume;
// Frequency of the test sound [Hz]
UPROPERTY(EditAnywhere, Category="Sound Source Properties")
float Frequency; // [Hz]
USoundWaveProceduralTest *SoundWaveProcedural;
// Begin USoundNode Interface
virtual int32 GetMaxChildNodes() const override;
virtual float GetDuration() override;
virtual void ParseNodes( FAudioDevice* AudioDevice, const UPTRINT NodeWaveInstanceHash, FActiveSound& ActiveSound, const FSoundParseParameters& ParseParams, TArray<FWaveInstance*>& WaveInstances ) override;
virtual FString GetUniqueString() const override;
#if WITH_EDITOR
virtual FString GetTitle() const override;
#endif
protected:
bool SoundWaveIsInitialized;
void CreateSoundWaveStreaming();
};
// EndFile: SoundNodeProceduralTest.h
// BeginFile: SoundNodeProceduralTest.cpp
#include "AudioTestProject.h"
#include "SoundNodeProceduralTest.h"
USoundNodeProceduralTest::USoundNodeProceduralTest(const class FPostConstructInitializeProperties& PCIP)
: Super(PCIP)
, Volume(0.5f)
, Frequency(100.0f)
{
}
int32 USoundNodeProceduralTest::GetMaxChildNodes() const
{
return 0;
}
float USoundNodeProceduralTest::GetDuration()
{
return INDEFINITELY_LOOPING_DURATION;
}
void USoundNodeProceduralTest::ParseNodes(FAudioDevice* AudioDevice, const UPTRINT NodeWaveInstanceHash, FActiveSound& ActiveSound, const FSoundParseParameters& ParseParams, TArray<FWaveInstance*>& WaveInstances)
{
// method 1 : works "sort of" , but seems like the clip stops playing and restarts
if(!SoundWaveIsInitialized)
{
CreateSoundWaveStreaming();
}
if(SoundWaveIsInitialized)
{
if(SoundWaveProcedural)
{
SoundWaveProcedural->Frequency = Frequency;
SoundWaveProcedural->Volume = Volume;
SoundWaveProcedural->Parse(AudioDevice, NodeWaveInstanceHash, ActiveSound, ParseParams, WaveInstances);
}
}
}
void USoundNodeProceduralTest::CreateSoundWaveStreaming()
{
SoundWaveProcedural = NewObject<USoundWaveProceduralTest>();
SoundWaveIsInitialized = true;
}
FString USoundNodeProceduralTest::GetUniqueString() const
{
return TEXT("FOO UNIQUE STRING");
}
#if WITH_EDITOR
FString USoundNodeProceduralTest::GetTitle() const
{
return TEXT("Procedural Sound Wave Source");
}
// EndFile: SoundNodeProceduralTest.cpp
// BeginFile:SoundWaveProceduralTest.h
#pragma once
#include "Sound/SoundWave.h"
#include "SoundWaveProceduralTest.generated.h"
//////////////////////////////////////////////////////////////////////////
/// USoundWaveProceduralTest :
/// A USoundWave that generates a single tone procedurally. This is
/// A test class that demonstrates the ability to create audio
/// procedurally (e.g. other algorithms could be used to generate
/// wind sounds etc.)
//////////////////////////////////////////////////////////////////////////
UCLASS()
class AUDIOTESTPROJECT_API USoundWaveProceduralTest : public USoundWave
{
GENERATED_UCLASS_BODY()
float Frequency; // [Hz]
// UObject interface
virtual void Serialize(FArchive& Ar) override;
virtual void GetAssetRegistryTags(TArray<FAssetRegistryTag>& OutTags) const override;
virtual SIZE_T GetResourceSize(EResourceSizeMode::Type Mode) override;
// USoundWave interface
virtual int32 GeneratePCMData(uint8* PCMData, const int32 SamplesNeeded) override;
virtual int32 GetResourceSizeForFormat(FName Format) override;
virtual FByteBulkData* GetCompressedData(FName Format) override;
virtual void InitAudioResource(FByteBulkData& CompressedData) override;
virtual bool InitAudioResource(FName Format) override;
private:
// Time of the last sample that was copied to the PCMData buffer [s]
float Time;
// Time between two consecutive samples in the audio signal [s]
float DeltaTime;
// Angular frequency of the single tone [rad/s]
float Omega;
};
#endif
// EndFile:SoundWaveProceduralTest.h
// BeginFile:SoundWaveProceduralTest.cpp
#include "AudioTestProject.h"
#include "SoundWaveProceduralTest.h"
USoundWaveProceduralTest::USoundWaveProceduralTest(const class FPostConstructInitializeProperties& PCIP)
: Super(PCIP)
, Time(0.0f)
, Frequency(100.0f)
{
SampleRate = 44100;
NumChannels = 1;
Duration = INDEFINITELY_LOOPING_DURATION;
bProcedural = true;
bLooping = false;
DeltaTime = 1.0f / SampleRate; // time increment between samples [s]
}
int32 USoundWaveProceduralTest::GeneratePCMData(uint8* PCMData, const int32 SamplesNeeded)
{
float TimeStart = Time;
Omega = 2.0f * PI * Frequency; // angular frequency [rad/s]
for(int i = 0; i < SamplesNeeded; i++)
{
Time = TimeStart + i * DeltaTime;
int32 a = FMath::RoundToInt( 65535.0f * (1.0f + FMath::Sin(Omega*Time)) * 0.5f);
if(a > 65535)
{
a = 65535;
}
else if( a < 0)
{
a = 0;
}
uint16 ua = (uint16)a;
PCMData[2*i] = ua; // Low Byte
PCMData[2*i+1] = ua >> 8; // High Byte
}
Time = Time + SamplesNeeded * DeltaTime;
int32 BytesProvided = SamplesNeeded * 2;
return BytesProvided;
}
SIZE_T USoundWaveProceduralTest::GetResourceSize(EResourceSizeMode::Type Mode)
{
return 0;
}
int32 USoundWaveProceduralTest::GetResourceSizeForFormat(FName Format)
{
return 0;
}
void USoundWaveProceduralTest::GetAssetRegistryTags(TArray<FAssetRegistryTag>& OutTags) const
{
// should never be in asset registry
check(false);
}
FByteBulkData* USoundWaveProceduralTest::GetCompressedData(FName Format)
{
return NULL;
}
void USoundWaveProceduralTest::Serialize(FArchive& Ar)
{
// do not call the USoundWave version of serialize
USoundBase::Serialize(Ar);
}
void USoundWaveProceduralTest::InitAudioResource(FByteBulkData& CompressedData)
{
// should never be pushing compressed data
check(false);
}
bool USoundWaveProceduralTest::InitAudioResource(FName Format)
{
//nothing to be done to initialize
return true;
}
// EndFile:SoundWaveProceduralTest.cpp
Hi - Thanks for replying! I have checked and re-checked that I’m filling in the buffer properly but maybe I missed something. I added full source code for the 2 classes to this thread and would be awesome if you can double check my GeneratePCMData code method. I agree that filling in the end (or beginning) of the buffer incorrectly, could lead to this “clicking” result. By adjusting the MONO_PCM_BUFFER_SAMPLES value I can hear the click move to exactly the expected time interval (44100 = 1 second, etc.), so it’s definitely something to do with the start/end of the buffer:
int32 USoundWaveProceduralTest::GeneratePCMData(uint8* PCMData, const int32 SamplesNeeded)
{
float TimeStart = Time;
Omega = 2.0f * PI * Frequency; // angular frequency [rad/s]
for(int i = 0; i < SamplesNeeded; i++)
{
Time = TimeStart + i * DeltaTime;
int32 a = FMath::RoundToInt( 65535.0f * (1.0f + FMath::Sin(Omega*Time)) * 0.5f);
if(a > 65535)
{
a = 65535;
}
else if( a < 0)
{
a = 0;
}
uint16 ua = (uint16)a;
PCMData[2*i] = ua; // Low Byte
PCMData[2*i+1] = ua >> 8; // High Byte
}
Time = Time + SamplesNeeded * DeltaTime;
int32 BytesProvided = SamplesNeeded * 2;
return BytesProvided;
}
I make a couple of assumptions:
The int32 returned by GeneratePCMData is the number of bytes placed into PCMData (e.g. NOT the number of samples).
The PCMData is 16 bits per sample (even though the argument passed to GeneratePCMData is uint8* PCMData).
I think I’m really close to getting this working. Thanks for your help!
Thanks Spaehling -
I doubt this is the root cause, but I will dig into it. I wrote a different streaming audio program completely outside of UE a few years ago (in C++ using WinMM). I will see if that program runs properly on this PC.
I added the source code for 2 classes to this thread. You’re welcome to try these on your PC !
It was a bug with settting the Time value after filling the buffer! (Thanks for the heads-up !). So each time it started filling in the PCM data it would be starting with the wrong time value. Using the corrected code line below fixes the issue and it’s playing correctly now in the Sound Cue (play Cue).
This code works now, but is a little sloppy as I was focused on getting the mechanism working properly first. (So clean it up a bit if you want to use it for something real)
int32 USoundWaveProceduralTest::GeneratePCMData(uint8* PCMData, const int32 SamplesNeeded)
{
float TimeStart = Time;
Omega = 2.0f * PI * Frequency; // angular frequency [rad/s]
for(int i = 0; i < SamplesNeeded; i++)
{
Time = TimeStart + i * DeltaTime;
int32 a = FMath::RoundToInt( 65535.0f * (1.0f + FMath::Sin(Omega*Time)) * 0.5f);
if(a > 65535)
{
a = 65535;
}
else if( a < 0)
{
a = 0;
}
uint16 ua = (uint16)a;
PCMData[2*i] = ua; // Low Byte
PCMData[2*i+1] = ua >> 8; // High Byte
}
//Time = Time + SamplesNeeded * DeltaTime; // INCORRECT LINE
Time = TimeStart + SamplesNeeded * DeltaTime; // CORRECTED LINE
int32 BytesProvided = SamplesNeeded * 2;
return BytesProvided;
}
i first wrote that it might not be your issue but then it really sounded exactly like the problems i had with making music. And it could have been since your making realtime sound.
The only thing that werent exact the same sounding where your clicks where i was experiencing more crackling sounds. ;D
But great that it worked out for you and thanks for the source. Maybe i try it out for making engine sounds.
@tegleg - your welcome , you should be able to fill in the buffer now with something more interesting!
@Spaehling - thanks for comments, all comments help move toward the root cause.
We also owe thanks to @ for urging me to take another look at that buffer to find the last bug.
One last detail: If you want to use this as a starting point for “something real”, you’ll need to clean it up a bit. One loose end is that I create the USoundWaveProceduralTest object using the “NewObject” construction and you’ll notice I don’t delete it anywhere or register for garbage collection. You’ll want to do the “right thing” for object destruction yourselves based on your projects (I’m still thinking about how I will approach this for my case).
hi folks, i know this is a bit late but I’m really interested in that topic.
have you managed to make it work on a level, or just within cue editor?
each time i put my sound cue in the map and hit play unreal crashes or exits with a ‘pure virtual method invoked’ message.
and for the record, the sound of this example is not a sin wave, or at least in my editor it doesn’t sound like one.
anyway, good job and thank you for the knowledge-base you already gave.
if only unreal engine had better documentation…
