You don’t need to use the packaged game produced by frontend.
You can cook the game yourself using the CookPackages commandlet (documentation is here: UDK | CommandletList).
UDK.exe CookPackages -platform=PC YourMap.udk YourMap.udk -fullAll Frontend is doing is just running these commandlets anyway.
Then once you’ve done that, simply copy the required files from your UDK install into an empty game install folder. You can create a .bat file to do this automatically for you.
You can reference the files created by the standard packaged installation to see which files you need to copy. You just need to make sure you copy the Win64 folder.