Hey Everybody!
I’ve been using the pixel streaming plug-in for a while now on various platforms ranging from desktop to mobile devices. I’ve also set up a STUN and TURN servers and connection seems to be established successfully across all types of networks. I did notice however that sometimes the stream would start frozen; let me explain.
After receiving the first frame, the video would freeze for 20 seconds, and then continue. The audio would keep playing normally in the meantime, and input events (mouse, keyboard) would still get registered and update the game. It happens 25% of the times after refreshing the page and immediately after pressing “play”.
I did notice that stream never actually stopped, the data kept flowing, but for some reason frames were not decoded:
In the chart, you can see how it decodes the first frame at 8:17 AM and then it freezes
I did notice that MediaStreamTrack.muted is true while the video is frozen, and then it becomes false once the stream continues.
I was looking for the meaning of the muted property, and according to w3 specification, it’s not controllable by the application, and it usually represents an underlying issue with the hardware or the user agent. Here’s a quote (see source):
A MediaStreamTrack is muted when the source is temporarily unable to provide the track with data. A track can be muted by a user. Often this action is outside the control of the application. This could be as a result of the user hitting a hardware switch or toggling a control in the operating system / User Agent chrome. A track can also be muted by the User Agent.
This lead me to think that there’s something with the pixel-streaming plug-in implementation on the Unreal side. Did anyone else experience that issue? Any thoughts about it?