Converting TArray<uint8> from OpenAI TTS API to USoundWave in Unreal Engine (No Paid Plugins)

Development Programming & Scripting C++
, , ,
1

I am working on integrating OpenAI’s Text-to-Speech (TTS) API with Unreal Engine to get audio responses in binary format. I am receiving the binary audio data in a TArray<uint8> format from FHttpResponsePtr, specifically after calling OpenAI’s API requesting audio in WAV format.

My goal is to take this binary data and convert it to a USoundWave object so that I can play it back in Unreal Engine using PlaySound2D. I want to avoid using any paid plugins and work with the built-in Unreal Engine functionality.

Here is what I currently have:

  1. I am able to receive the TArray<uint8> audio data from the API call.
  2. I want to stream or convert this audio data into USoundWave in a way that allows it to be played directly within Unreal Engine.

Questions:

  • How can I efficiently convert the TArray<uint8> audio data from FHttpResponsePtr into a USoundWave?
  • What would be the most suitable audio format (PCM, WAV, etc.) to request from the API that can easily be converted to USoundWave without needing external plugins?

Thanks in advance!

2

Hi, if you don’t want to use external plugins, check out USoundWaveProcedural, you can queue your own PCM audio data and play it back, and PCM would probably be the best format for that. But if you want better compression and faster audio transfer from the OpenAI TTS API, you might want to go with e.g. MP3 or OGG Vorbis and decode the audio manually.