Pixel Streaming — how to stream audio without playing it locally?

Development Programming & Scripting
, ,
1

Hi everyone,

I’m working on a Pixel Streaming setup in UE5 and I need to send audio to the stream without it being reproduced locally on the server machine.

Is there any way to achieve this?

Thanks!

2

Hey @Feanor :waving_hand:

As far as I’m aware, this functionality is not possible using the old Pixel Streaming plugin.

However, if you are able to use Pixel Streaming 2, then this can easily be achieved.

#include "IPixelStreaming2AudioProducer.h"

class FMyAudioProducer : public IPixelStreaming2AudioProducer
{
public:
    FMyAudioProducer() = default;
    virtual ~FMyAudioProducer() = default;
};


/**
* Execute this block wherever you see fit.
*/
// Obtain a TSharedPtr<IPixelStreaming2Streamer>. This can be either the default streamer obtained from the IPixelStreaming2Module or a custom streamer if you are using that
TSharedPtr<IPixelStreaming2Streamer> Streamer = /* Obtain somehow */;
TSharedPtr<FMyAudioProducer> AudioProducer = MakeShared<FMyAudioProducer>();

Streamer->AddAudioProducer(AudioProducer);

/**
* Now all that is required for your audio producer to stream audio is to call AudioProducer->PushAudio(const float* InBuffer, int32 NumSamples, int32 NumChannels, int32 SampleRate)
*
* Obtaining audio is outside the scope of this example and is up to you depending on what audio you want to stream
*/

Hope this helps!

Cheers

3

Hi,

Assuming this is for a chatbot or AI avatar use case, I would actually recommend not sending the user’s audio directly into Unreal Engine.

For the best UX and lowest latency, you can handle the audio input on the frontend instead. With Pixel Streaming, the frontend can be customized, so the user’s microphone input can be captured there and sent directly to your AI model or backend service. Once the AI generates a response, you can send the response back to the Unreal Engine application, where your character or AI avatar can speak it.

This avoids routing the audio through the Unreal server unnecessarily and can make the input-to-response flow much faster.

If you want to experiment with this kind of frontend customization, you can use streampixel.io or any Pixel Streaming provider that supports custom frontend workflows. StreamPixel can be especially cost-effective for longer experiences since it does not charge per minute.