Pause / Un-Pause with same key not working

If I remove the Set Input Modes entirely I get the desired result but then can no longer use the mouse to interact with the menu.