we have a Pixel Streaming Build up and running an are so far quite happy with the results but we have a big problem getting a good audio signal using pixel streaming. It is really low quality, mono only and we have no idea where to look, to change the settings to something better. Has someone experienced something similar and has any hints where to look, because we skimmed through the source code but couldn’t find the files, where this settings are configured.
Hi Tyrus86,
Yes, I had a chance to try it today with a sample project, and indeed it seems it’s mono and rather low bitrate.
I’ll have to to chase some colleague experienced with WebRTC internals to try and debug this, since I can’t find anything obvious on our side.
Looks like we found the issue.
A fix will be available in a future version of the engine, but if you want to try it out, you need to change two files:
Engine\Plugins\Media\PixelStreaming\Source\PixelStreaming\Private\Streamer.cpp, change the FStreamer::AddStreams method to make use of a cricket::AudioOptions struct. Full function follows:
void FStreamer::AddStreams(FPlayerId PlayerId)
{
const FString StreamId = TEXT("stream_id");
const char AudioLabel[] = "audio_label";
const char VideoLabel[] = "video_label";
FPlayerSession* Session = GetPlayerSession(PlayerId);
check(Session);
cricket::AudioOptions AudioSourceOptions;
AudioSourceOptions.echo_cancellation = false;
AudioSourceOptions.auto_gain_control = false;
AudioSourceOptions.noise_suppression = false;
if (bPlanB)
{
rtc::scoped_refptr<webrtc::MediaStreamInterface> Stream;
if (auto* StreamPtr = Streams.Find(StreamId))
{
Stream = *StreamPtr;
}
else
{
Stream = PeerConnectionFactory->CreateLocalMediaStream(TCHAR_TO_ANSI(*StreamId));
rtc::scoped_refptr<webrtc::AudioTrackInterface> AudioTrackLocal(
PeerConnectionFactory->CreateAudioTrack(AudioLabel, PeerConnectionFactory->CreateAudioSource(AudioSourceOptions)));
Stream->AddTrack(AudioTrackLocal);
auto VideoCapturerStrong = std::make_unique<FVideoCapturer>(HWEncoderDetails);
VideoCapturer = VideoCapturerStrong.get();
rtc::scoped_refptr<webrtc::VideoTrackInterface> VideoTrackLocal(PeerConnectionFactory->CreateVideoTrack(
VideoLabel, PeerConnectionFactory->CreateVideoSource(std::move(VideoCapturerStrong))));
Stream->AddTrack(VideoTrackLocal);
Streams[StreamId] = Stream;
}
verifyf(Session->GetPeerConnection().AddStream(Stream), TEXT("Failed to add stream for player %u"), PlayerId);
}
else
{
if (!Session->GetPeerConnection().GetSenders().empty())
{
return; // Already added tracks
}
if (!AudioTrack)
{
AudioTrack =
PeerConnectionFactory->CreateAudioTrack(AudioLabel, PeerConnectionFactory->CreateAudioSource(AudioSourceOptions));
}
if (!VideoTrack)
{
auto VideoCapturerStrong = std::make_unique<FVideoCapturer>(HWEncoderDetails);
VideoCapturer = VideoCapturerStrong.get();
VideoTrack = PeerConnectionFactory->CreateVideoTrack(
VideoLabel, PeerConnectionFactory->CreateVideoSource(std::move(VideoCapturerStrong)));
}
auto Res = Session->GetPeerConnection().AddTrack(AudioTrack, { TCHAR_TO_ANSI(*StreamId) });
if (!Res.ok())
{
UE_LOG(PixelStreamer, Error, TEXT("Failed to add AudioTrack to PeerConnection of player %u. Msg=%s"), Session->GetPlayerId(), ANSI_TO_TCHAR(Res.error().message()));
}
Res = Session->GetPeerConnection().AddTrack(VideoTrack, { TCHAR_TO_ANSI(*StreamId) });
if (!Res.ok())
{
UE_LOG(PixelStreamer, Error, TEXT("Failed to add VideoTrack to PeerConnection of player %u. Msg=%s"), Session->GetPlayerId(), ANSI_TO_TCHAR(Res.error().message()));
}
}
}
Engine/Source/Programs/PixelStreaming/WebServers/SignallingWebServer/scripts/webRtcPlayer.js, change the handleCreateOffer function to:
handleCreateOffer = function (pc) {
pc.createOffer(self.sdpConstraints).then(function (offer) {
offer.sdp = offer.sdp.replace("useinbandfec=1", "useinbandfec=1;stereo=1;maxaveragebitrate=128000");
pc.setLocalDescription(offer);
if (self.onWebRtcOffer) {
// (andriy): increase start bitrate from 300 kbps to 20 mbps and max bitrate from 2.5 mbps to 100 mbps
// (100 mbps means we don't restrict encoder at all)
// after we `setLocalDescription` because other browsers are not c happy to see google-specific config
offer.sdp = offer.sdp.replace(/(a=fmtp:\d+ .*level-asymmetry-allowed=.*)\r\n/gm, "$1;x-google-start-bitrate=10000;x-google-max-bitrate=20000\r\n");
self.onWebRtcOffer(offer);
}
},
function () { console.warn("Couldn't create offer") });
}
