Heya @J_Panov,
Very interesting use case you have here.
Short answer: This cannot be achieved with PixelStreaming2 in 5.6 as there were bugs with the public API. If possible, I’d suggest updating to 5.8 as there were a few fixes specifically targeting public API fixes. If that is possible, see the rest of the answer for how to achieve your desired functionality.
PixelStreaming2 has no built-in per-submix routing. Fortunately, you shouldn’t need to modify the plugin source to get what you want as there’s extension points by way of IPixelStreaming2AudioProducer and IPixelStreaming2Streamer::AddAudioProducer that are designed for pushing your own per-streamer audio.
Root cause
By default, every streamer listens to the main engine submix. In AudioCapturer.cpp:53-58 the capturer creates one FAudioProducer per audio device, and that producer registers a submix buffer listener against the main submix specifically — see AudioProducer.cpp:16-22:
AudioDevice->RegisterSubmixBufferListener(Listener.ToSharedRef(), AudioDevice->GetMainSubmixObject());
This “engine” producer is always created and there is currently no flag to disable it. So as long as your English/Spanish audio reaches the master submix, all streamers will pick it up. That’s the root of the bleed-through.
Two part workaround
Part 1 (audio routing — your side): Configure your per-language Submixes so they do not route up into the Master submix. The simplest way is to set the parent/output of each language submix so it isn’t mixed into Master (e.g. route it to an isolated submix that doesn’t feed Master, or zero its master send). If the language audio still hits the Master submix, the default engine producer above will leak it into every stream and nothing downstream can un-mix it.
Part 2 (per-streamer injection): For each streamer, add a custom IPixelStreaming2AudioProducer that listens to one submix and pushes its buffers in. Per-producer audio is mixed only into the streamer it was added to.
The relevant public API:
Internally PS2 wires a custom producer up exactly the same way the engine producer is wired, which subscribes to your OnAudioPushed event. So all you have to do is implement the producer and call PushAudio whenever your submix generates a buffer.
A single class that is both a submix buffer listener and a PS2 audio producer:
#include "IPixelStreaming2AudioProducer.h"
#include "ISubmixBufferListener.h"
// Bridges one USoundSubmix -> one PS2 streamer.
class FSubmixAudioProducer
: public IPixelStreaming2AudioProducer // PS2 side: we call PushAudio()
, public ISubmixBufferListener // Engine side: receives submix buffers
{
public:
static TSharedPtr<FSubmixAudioProducer> Create(USoundSubmix* Submix)
{
TSharedPtr<FSubmixAudioProducer> Producer = MakeShared<FSubmixAudioProducer>();
if (FAudioDeviceManager* DM = FAudioDeviceManager::Get())
{
if (FAudioDevice* Device = DM->GetActiveAudioDevice().GetAudioDevice())
{
Producer->BoundDevice = Device;
Producer->BoundSubmix = Submix;
// Listen to THIS submix only, not the master submix.
Device->RegisterSubmixBufferListener(Producer.ToSharedRef(), Submix);
}
}
return Producer;
}
void Unregister()
{
if (BoundDevice && BoundSubmix.IsValid())
{
BoundDevice->UnregisterSubmixBufferListener(AsShared(), BoundSubmix.Get());
BoundDevice = nullptr;
}
}
// ISubmixBufferListener — called on the audio thread when the submix renders a buffer.
virtual void OnNewSubmixBuffer(const USoundSubmix* OwningSubmix, float* AudioData,
int32 NumSamples, int32 NumChannels, const int32 SampleRate, double /*AudioClock*/) override
{
// Forward straight into the PS2 pipeline. PS2 handles resample/mix downstream.
// NumSamples here is the total interleaved count; PS2 expects per-channel frames,
// so divide out the channels.
PushAudio(AudioData, NumSamples / NumChannels, NumChannels, SampleRate);
}
// Required by ISubmixBufferListener for log/debug identification.
virtual const FString& GetListenerName() const override
{
static const FString Name = TEXT("PixelStreaming2SubmixProducer");
return Name;
}
private:
FAudioDevice* BoundDevice = nullptr;
TWeakObjectPtr<USoundSubmix> BoundSubmix;
};
Wiring it up, one submix per streamer:
void RouteSubmixToStreamer(const FString& StreamerId, USoundSubmix* LanguageSubmix)
{
IPixelStreaming2Module& Module = IPixelStreaming2Module::Get();
TSharedPtr<IPixelStreaming2Streamer> Streamer = Module.FindStreamer(StreamerId);
if (!Streamer || !LanguageSubmix)
{
return;
}
TSharedPtr<FSubmixAudioProducer> Producer = FSubmixAudioProducer::Create(LanguageSubmix);
// Audio from THIS producer is mixed only into THIS streamer.
Streamer->AddAudioProducer(Producer);
// Keep the TSharedPtr alive for as long as you want the routing active.
// On teardown: Streamer->RemoveAudioProducer(Producer); Producer->Unregister();
}
// e.g.
// RouteSubmixToStreamer(TEXT("Streamer1"), EnglishSubmix);
// RouteSubmixToStreamer(TEXT("Streamer2"), SpanishSubmix);
Things to look out for
Things to watch
- Keep the producer alive.
AddAudioProducer stores a TSharedPtr, but you should hold your own reference too and call RemoveAudioProducer + your Unregister on shutdown to detach the submix listener cleanly.
OnNewSubmixBuffer runs on the audio thread — keep it allocation-free (the snippet just forwards the pointer, which is fine).- The default master-submix producer is still active. Part 1 is not optional — if your language audio reaches Master, it leaks into every stream regardless of this code. Verify isolation by muting one language and confirming the other stream goes silent.
- Sample-rate/channel mismatches are handled downstream — PS2’s
FAudioProducer resamples/mixes to the WebRTC target, so you can push the submix’s native format.
Let me know how it goes!