macOS Fullscreen Deadlock in CocoaWindow.cpp

macOS Fullscreen Deadlock in CocoaWindow.cpp

Hit a deadlock on Mac when clicking the green fullscreen button. The issue is in CocoaWindow.cpp (https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Source/Runtime/ApplicationCore/Private/Mac/CocoaWindow.cpp).

What happens:

  1. User clicks the green fullscreen button
  2. AppKit calls windowWillEnterFullScreen:
  3. The fullscreen transition triggers a resize or main-window change
  4. windowWillResize:toSize: or windowDidBecomeMain: fires — both call GameThreadCall(^{…}, true, …) with wait=true, blocking the main thread until the game thread processes the block
  5. The game thread is waiting on a deferred fullscreen event or main-thread state update to continue
  6. Deadlock — main thread waits on game thread, game thread waits on main thread

The root cause is that windowWillResize: (line ~330) and windowDidBecomeMain: (line ~197) both use blocking GameThreadCall with wait=true. During a fullscreen transition, AppKit holds internal locks on the main thread when it calls these delegates, so the main thread can’t pump messages while waiting for the game thread — and the game thread can’t proceed without the main thread.