If Epic hasn’t changed anything lately, you can’t build a dedicated server via Launcher Version. You will only get the required Build Configs if you build the Engine from Source (like you said)
Yes, thats true and still ridiculous in my opinion. You need an engine source build to be able to generate a dedicated server. So, sooner or later all or most multiplayer games will need a source build.