Windows Live Coding Duplex Pipe Deadlock

`DuplexPipe::Read` and `DuplexPipe::Send` synchronously read and write to named pipes. This is a blocking operation and can cause the Unreal Editor to deadlock if executed at the same time, a known limitation in the Windows operating system. Without a proper Mutex, this means that Live Coding can inherently cause the Editor to hang indefinitely.

This occurs most commonly running Editor commandlets, where the Live Coding module teardown calls to read the pipe (via `SendCommandAndWaitForAck` in `ClientStartupThread::~ClientStartupThread`) at the same time as it may be in the process of writing the same pipe. A good repro case is using the Command Line Rendering support as described in: https://dev.epicgames.com/documentation/en-us/unreal-engine/using-command-line-rendering-with-move-render-queue-in-unreal-engine?application_version=5.6

Related, since the LiveCoding settings are stored in the `EditorPerProjectUserSettings`, this cannot be toggled effectively via .ini at the project or engine level and must instead be set via TargetSettings, giving less flexibility when turning this feature off to prevent deadlocking.

Hi Kathy,

We have also come across this issue with Live Coding internally.

We are currently working on integrating a new version of Live Coding which will avoid this issue since it is not activated until you first use the live coding keyboard shortcut as opposed to the old version which was started on editor load.

The integration is still work in progress so definitely experimental at this stage, but if you want to give it a try it can be enabled by changing:

Engine/Source/Developer/Windows/LiveCoding/LiveCoding.Build.cs

static bool UseLiveCoding2 = false;

to be true.

You can then perform live editing through the Ctrl-Alt-F11 shortcut.

Hope this helps.

I cannot speak for whether the new version of Live Coding directly addresses this issue since it is actually an integration of Live++.

However, since it’s now only activated on the first time you use the keyboard shortcut you will not see the issue for Editor commandlets.

Will this new version of Live Coding also address the underlying race condition with using named pipes that causes this deadlock to occur?