How to detect whether keyboard/mouse or controller was interacted with last

From looking into the code: the UGameViewportClient handles inputs differently when you set FInputModeUIOnly. So the solution would be to edit the UGameViewportClient to catch the last used controller and pass the information down from there.