What you want to do is run a dedicated server on your local development PC for testing. You can then point the UE4 client on your Android devices to the external IP of your local development PC. You will need to do the following to set this up:
Setup port forwarding to forward ports in the 7777 to 7778+ range to your local development PC
Add firewall rules to Windows firewall for this range
Start your project (you don’t have to compile a dedicated server) with the UE4 Editor using a bat file like this:
C:\Program Files\Epic Games\UE_4.18\Engine\Binaries\Win64\UE4Editor.exe “C:\Users[YourUserName]\Documents\Unreal Projects\OpenWorldStarterTest\OpenWorldStarter.uproject” MapName?listen -server -log -nosteam -messaging -port=7778
You will have to replace some sections in the above to get it to work, but that is the idea.