I found a solution to this problem in the Unity forums.
Some devs had the exact some issue and they fixed it by disabling the Google VR Service on their devices.
For me this worked like a charm for pretty much all UE4 apps that were having this problem before.