We use Live Link hub on virtual production stages because it helps rebroadcast subjects that are either connected directly to an operator machine, or unicasting to that machine.
The problem is that we’ve had stability issues with using it. Just opening up Hub with it blank seems to be stable, but when you start to add subjects, connect multiple clients to it. You can see the memory usage slowly creep up over time, and eventually crash with out of memory rendering a resource error, a log is attached.
While in some cases it’s taken hours to do so, in other cases we’ve had it crash frequently even in the middle of shooting.
As first we assumed that a third party plugin (the DCS plugin) may be to blame, but it’s been crashing even with a single connection to tracking software like optitrack.
Steps to Reproduce
Start up Live Link Hub, Add some source such as from Optitrack, launch an nDisplay Cluster and have it connect to Hub.
Shut down the cluster, and then leave Hub running, eventually, it’s going to crash with an out of memory error.
Hey Matthew,
We’ve been aware of this issue for some time.. Unfortunately it seems like this is caused by a bug in the d3d12 driver, where GPU submissions builds up when the app window is minimized. The D3D12::ExecuteCommandLists calls get progressively slower, which causes memory to creep up until the process goes OOM.
At the moment, we don’t have a clear workaround besides keeping LLH unminimized. We’ve escalated the issue with NVidia, but we’re still waiting to hear from them.
I’ll follow up on this thread as soon as they get back to us.
Hey Matthew, good timing, NVidia just released the fix, should be available in 581.03 or later. So far it seems to behave correctly according to my testing.
Give it a try when you can and let me know if you still crash after updating.
Thank you for the quick responses.
Not minimizing is a temporary workaround until we get drivers updated.