How to execute gameplay code from the audio thread

Hi folks, I’m writing a plugin that lets you execute gameplay using your microphone.
Full source code available on GitHub.

I’m using delegates, my idea was to bind gameplay code to a delegate and execute it from the audio thread.
Here’s my audio input subsystem

// ... 
	void UAudioInputSubsystem::Initialize(FSubsystemCollectionBase& Collection)
{	
	Audio::FAudioCaptureDeviceParams Params;
	Params.DeviceIndex = 0;
	
	EnvelopeFollowerInitParams.AttackTimeMsec = 0.5f;
	EnvelopeFollowerInitParams.ReleaseTimeMsec = 1.0f;
	EnvelopeFollowerInitParams.AnalysisWindowMsec = 2.0f;
	EnvelopeFollower = Audio::FEnvelopeFollower(EnvelopeFollowerInitParams);
	
	const FOnSoundDelegate& Delegate = OnSoundDelegate;

	Audio::FOnCaptureFunction OnCapture = [this, Delegate](const float* AudioData, int32 NumFrames, int32 NumChannels, int32 SampleRate, double StreamTime, bool bOverFlow)
	{
		EnvelopeFollower.ProcessAudio(AudioData, NumFrames);
		const TArray<float>& EnvelopeValues = EnvelopeFollower.GetEnvelopeValues();

		for (const float Value : EnvelopeValues)
		{
			const float Amplitude = FMath::Abs(Value);
			if (Amplitude > FilterThreshold)
			{
				FAudioThread::RunCommandOnGameThread([Delegate, Amplitude]()
				{
					Delegate.Broadcast(Amplitude);
				});
			}
		}
	};

	bool IsOpen = AudioCapture.OpenCaptureStream(Params, MoveTemp(OnCapture), AudioBufferSize);
	
	if (IsOpen)
	{
		AudioCapture.StartStream();
	} 
}

And here’s how I bind the gameplay code to the delegate in a game mode

void AVoicerinoGameModeBase::BeginPlay()
{
	const auto GameInstance = UGameplayStatics::GetGameInstance(GetWorld());
	UAudioInputSubsystem* AudioInput = GameInstance->GetSubsystem<UAudioInputSubsystem>();
	AudioInput->OnSoundDelegate.AddDynamic(this, &AVoicerinoGameModeBase::InputReceived);
}

void AVoicerinoGameModeBase::InputReceived(float Amplitude)
{
	DrawDebugPoint(GetWorld(), FVector(0,0, 50.f), 30.f, FColor::White);
	UNiagaraFunctionLibrary::SpawnSystemAtLocation(GetWorld(), NiagaraSystemToSpawnOnAudioInputReceived, FVector());
}

It works if I log to the console, but crashes if I try to spawn a particle system (e.g. anything gameplay thread related). How can I call gameplay code without getting this crash?

The easy way would be to set a flag somewhere that is read by something running during a normal Tick(), which then responds to the flag being set by performing the required in game task.

1 Like

I changed it to a polling implementation, what do you think?