Help me with using Sound Generator Please...

Development Audio
, , ,
1

I’m planning to make a DJ Controller but I don’t think I understood how audio mixer works in UE well enough.

Well I can get PCM data from wav files and make that as sound wave and play it. that’s no problem at all.

What I want to do is to make my own circular buffer and feed my TArray to the aadio mixer so I have a full control over playback frame, time stretch, pitch shifting and all kinds of dsp stuff.

After a bit of research, I’m pretty positive that this thing is possible but I think I’m missing something.

Here’s what I’ve tried.
Basically I copied most of codes from SynthComponentToneGenerator.
I can utilize TArray that contains 32 bit interleaved pcm data clamped between -1.f to 1.f
this component is under game mode and I tried to do something like this:

image
image739×293 57.2 KB

this is my .h file

// Copyright Toby Kim, ETNZ. All Rights Reserved.

#pragma once

#include "CoreMinimal.h"
#include "Components/SynthComponent.h"
#include "Sound/SoundBase.h"
#include "Sound/SoundGenerator.h"
#include "DJSlot.generated.h"


class FDJGenerator : public ISoundGenerator
{
public:
	FDJGenerator(int32 InSampleRate, int32 InNumChannels);
	virtual ~FDJGenerator();

	//~ Begin FSoundGenerator 
	virtual int32 GetNumChannels() { return NumChannels; };
	virtual int32 OnGenerateAudio(float* OutAudio, int32 NumSamples) override;
	//~ End FSoundGenerator

	void GetAudioDataFromSynthComponent(const TArray<float>& inData);

private:
	int32 NumChannels = 2;
	
};

UCLASS(ClassGroup = Synth, meta = (BlueprintSpawnableComponent))
class MYPROJECT2_API UDJSlot : public USynthComponent
{
	GENERATED_BODY()

	UDJSlot(const FObjectInitializer& ObjInitializer);
	virtual ~UDJSlot();
		
	// Called to generate more audio
	virtual int32 OnGenerateAudio(float* OutAudio, int32 NumSamples) override;

public:	

	virtual ISoundGeneratorPtr CreateSoundGenerator(const FSoundGeneratorInitParams& InParams);


	UFUNCTION(BlueprintCallable, Category = "DJMachine")
		void GetAudioDataFromBP(const TArray<float>& inData) 
	{
		UE_LOG(LogTemp, Warning, TEXT("Valid ::  %s"), DJSoundGen.IsValid() ? TEXT("True") : TEXT("False"));
		AudioData.Empty();
		for (size_t i = 0; i < inData.Num(); i++)
		{
			AudioData.Add(inData[i]);
		}
		TotalNumSample = AudioData.Num();
		BufferIsEmpty = false;
	};

private:

	ISoundGeneratorPtr DJSoundGen;

	int16 WritePointer = 0;
	int16 ReadPointer = 0;
	int16 GlobalPointer = 0;
	int16 TotalNumSample;

	TArray<float> AudioData;
	TArray<float> ShortFFTBuffer;
	TArray<float> WriteBuffer;
	TArray<float> ReadBuffer;

	bool BufferIsEmpty = true;
};

and this is my cpp file.

// Copyright Toby Kim, ETNZ. All Rights Reserved.


#include "DJSlot.h"

FDJGenerator::FDJGenerator(int32 InSampleRate, int32 InNumChannels)
	: NumChannels(InNumChannels)
{

}
FDJGenerator::~FDJGenerator()
{
}
int32 FDJGenerator::OnGenerateAudio(float* OutAudio, int32 NumSamples)
{
	UE_LOG(LogTemp, Warning, TEXT("Please Callme"));
	return int32();
}

void FDJGenerator::GetAudioDataFromSynthComponent(const TArray<float>& inData)
{
	
}

// Sets default values
UDJSlot::UDJSlot(const FObjectInitializer& ObjInitializer)
	: Super(ObjInitializer)
{
	NumChannels = 2;
}

UDJSlot::~UDJSlot()
{
}


int32 UDJSlot::OnGenerateAudio(float* OutAudio, int32 NumSamples)
{
	UE_LOG(LogTemp, Warning, TEXT("Please Callme2"));
	check(NumChannels != 0);

	if (GlobalPointer >= TotalNumSample) {
		return 0;
	}

	if (!BufferIsEmpty) {

		const int32 NumFrames = NumSamples;
		int32 SampleIndex = 0;

		for (int32 FrameIndex = 0; FrameIndex < NumFrames; ++FrameIndex)
		{
			OutAudio[FrameIndex] = AudioData[FrameIndex + GlobalPointer];
		}
		GlobalPointer += NumSamples;
	}
	return NumSamples;
}
ISoundGeneratorPtr UDJSlot::CreateSoundGenerator(const FSoundGeneratorInitParams& InParams)
{
	return DJSoundGen = ISoundGeneratorPtr(new FDJGenerator(InParams.SampleRate, InParams.NumChannels));
}

So, for a common ToneGenerator, if I just hook up “Start”, then It makes sounds but my component just doesn’t do anything. In fact, soundgenerator in my component wasn’t even valid, so I moved all the variables under the synth component. that’s why soundGenerator part in my code is empty.

What differs my code to ToneGenerator? I really don’t get how audio works in this engine.
if there’s anyone who can help me, they would be my life saver.