Distributing custom build to team

@Moss

Well we are hoping not to have much changes in the engine, but we do have some. So the idea is to separate engine changes into a separate branch that only engine programmers will work on and have the distribution for artists and game programmers in another location. When just building (non-rocket), I noticed that there are some dependent files that get removed from a temp directory that causes the engine to build when first compiling the game. The rocket build packages it all up and I donā€™t have to worry about what dependencies are needed. One main goal is to prevent game programmers from having to build the engine and not worry about accidentally hitting the rebuild or clean from the solution. So itā€™s a very convenient way of getting a distribution out for the team; setting up our build machine to pull source changes, build and submit.

Plus itā€™s kind of cool figuring out epics build process.

The rebuild build button without a confirmation is beyond evil :smiley: We have some own engine modules that contain common functionality that we reuse, so there is where we got a ton of changes.

@Kalms

I tried your addition and was successful in building a local build up to the point where it tried building the DerivedDataCache. At that point, the editor cmd failed because it could not load a DLL that it was looking in a VS2013 folder. Apparently the build script only copies the VS2015 folders in ThirdParty binaries, even though all my environment is supposedly 2013. By manually copying the VS2013 folders, I managed to make it work. Not sure how I can force the whole rocket process to assume VS2013. I also had to explicitly add a ThirdParty folder that was specific to my build, but I donā€™t see where the list of ThirdParty things to be included in Rocket build is.

Any pointers?

Edit: found the file InstalledEngineFilters.ini which seems the likely culprit

@lekjart

Iā€™m guessing you are on 4.10. I havenā€™t touched that version yet. However, if you donā€™t need to do the DDC, you can skip it by adding the -NoDDC parameter to the command line.
You can read this to see if you need to build it or not: Derived Data Cache | Unreal Engine 5.2 Documentation

Thanks, I think I got it figured out now. Mostlyā€¦

Only thing that I havenā€™t really managed is to redo a rocket build after a succesful one. There must be some config file somewhere that tells the build system that it has already created a rocket build and it doesnā€™t go into a full rebuild because of it. I tried deleting the LocalBuild folder and the Saved/Rocket folder but that only results in errors when I run the RunUAT command. I used forceincrementalcompile, so I was hopeful I wouldnā€™t need to do a full delete and cloning to get back doing a build after changing a few files. I am sure itā€™s possible but I canā€™t figure it out. Anyone has done that successfully?

Well the below contains some chicken scratch notes I took as I went through this months ago, not quite sure how accurate they are or if I am missing anything.

  • The ā€œ-CleanLocalā€ parameter deletes the manifests from Saved\GUBP directory. Those manifests determine whether a node script has been completed or not. If a manifest is there it will skip I think. So This might be all you need, but it will do a full build.
  • The ā€œ-ForceIncrementalCompileā€ parameter prevents the build from cleaning.
    • A big caution, if I remember correctly I think it was this parameter that will complain if binaries are compiled in non-rocket before running the rocket build again.

The incremental build with minimal changes is also on my TODO list. I did write up a custom GatherRocket node script to not copy the entire rocket build over and just the changes. However, there are more files that compiled that were unnecessary from a simple change.

This is what my thoughts on it (Just a reminder, Iā€™m still using version 4.8.2):
Each node script has a list of dependent node scripts and a DoBuild() method. If you pass in ā€œ-OnlyNodeā€, the dependencies are skipped.
The node GUBP.GamePlatformMonolithicsNode (I think the name of the node is ā€œUE4_Win64_Mono_Precompiledā€) compiles in the DoBuild():
+ UE4Game Development.
The node GUBP.ToolsNode ( I think the name of the node is ā€œToolsā€) compiles in the DoBuild():
+ CrashReport (shipping)
+ UE4EditorServices (development)
+ UnrealCEFSubProcess (Development)
+ UnrealFileServer (Development)
+ UnrealLightMass(Development)
+ UnrealPak(Development)
+ UnrealVersionSelector(Shipping)
+ BootstrapPackage(Shipping)

I havenā€™t tried it, but I was thinking of writing a shell script that only removes those single node manifests and running those nodes separately with ā€œ-OnlyNodeā€ and ā€œ-ForceIncrementalCompileā€, then also run my custom GatherRocket node script I mentioned earlier to package the changes.

There might be an easier way to do that. lol. I just havenā€™t gotten back into the build process yet.

Bumping this too as I have the same issue. If anyone from Epic is watching this, it would be extremely helpful if you posted some instructions on how to set up an environment, and what the process is to reproduce your launcher distributed builds. I work for a middleware vendor who maintain a UE4 integration, and our aim is to produce a Rocket build which we can push downstream to the subset of our customers who donā€™t modify engine source, and just want to have the off-the-shelf experience they get from the launcher distributed vanilla builds, but with our special sauce. Any help much appreciated!

And I build an engine based on various github forks that provide extra functionalities. And some of those forks have additional APIs and API keys which are restricted to my own machine.

We have a team of developers across two continents, each developer is using its own version for his/her own little specialty, then I merge the whole back into my engine. A custom build would be a boon to our development.

Hi guys, just a heads up, this works perfectly with 4.11;

I didnā€™t expect but it bundles custom plugins too.

12Gb output 7zips down to 3Gb.

How to do Rocket build is also shown in this Answerhub thread. It actually ends up being pretty simple. Iā€™ve had to make two quick changes for it to use VS2015 for PDBCopy as part of the process, and had to add the folders for the Templates as shown in the thread otherwise you may get an error. But once you do that itā€™s one command line call and looks identical to the Epic launcher builds.

This also works in 4.11:
https://answers.unrealengine.com/questions/393280/packaging-ue4-for-distributing-internally.html

Anybody have problems with build 4.11.2 and using it?
Mine builds fine, but when I try to generate projects I got error

GetBuildPlatform: No BuildPlatform found for Win64

Any clues why this happens? Source build works fine.

PS. Ok, i fixed it by making -CleanLocal build

Hi all,

First of all, thanks to all who have contributed to working out how to get a Rocked build made. We donā€™t plan to make many changes to the Unreal Engine as a whole, but would like to be able to distribute a custom build to the team with Win64 + PS4 + Xbox One support.

Iā€™ve managed to create a rocket build for distribution as per instructions on this thread with 2 target platforms Win32 + Win64. Now we also have access to PS4 and in future Xbox One portions of the Unreal Engine and Iā€™ve added both those platforms to -TargetPlatforms=Win32+Win64+PS4+XboxOne but it still only generates a rocket build that only create Win32 + Win64 cooked builds. Is it possible to add the Xbox One and PS4 build targets to a rocket build and if so, how do you do this because adding the extra targets to -TargetPlatforms command-line parameter doesnā€™t seem to do anything extra?

This can be done but it requires source code modification. I suggest you take the question to the appropriate forums. (sorry, I do not have access to those.)

GUBP.ExecuteNode: System.IO.FileNotFoundException: ęœŖčƒ½ę‰¾åˆ°ę–‡ä»¶ā€œD:\prg\UnrealEngine\Engine\Build\InstalledEngineFilters
.iniā€ć€‚
ę–‡ä»¶å:ā€œD:\prg\UnrealEngine\Engine\Build\InstalledEngineFilters.iniā€
åœØ System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
åœØ System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShar
e share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolea
n useLongPath, Boolean checkHost)
åœØ System.IO.FileStreamā€¦ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileO
ptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
åœØ System.IO.StreamReaderā€¦ctor(String path, Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 buffe
rSize, Boolean checkHost)
åœØ System.IO.StreamReaderā€¦ctor(String path, Encoding encoding)
åœØ System.IO.File.InternalReadAllLines(String path, Encoding encoding)
åœØ AutomationTool.FileFilter.ReadRulesFromFile(String FileName, String SectionName, String] AllowTags) 位ē½® D:\prg\U
nrealEngine\Engine\Source\Programs\AutomationTool\AutomationUtils\FileFilter.cs:č”Œå· 216
åœØ Rocket.StripRocketNode.DoBuild(GUBP bp) 位ē½® d:\prg\UnrealEngine\Engine\Source\Programs\AutomationTool\Scripts\Roc
ketBuild.Automation.cs:č”Œå· 447
åœØ AutomationTool.LegacyNode.DoBuild() 位ē½® d:\prg\UnrealEngine\Engine\Source\Programs\AutomationTool\GUBP\Nodes\Lega
cyNode.cs:č”Œå· 81
åœØ GUBP.ExecuteNode(BuildNode NodeToDo, Boolean bFake) 位ē½® d:\prg\UnrealEngine\Engine\Source\Programs\AutomationTool
\GUBP\GUBP.cs:č”Œå· 1545
Program.Main: AutomationTool exiting with ExitCode=1 (Error_Unknown)
BUILD FAILED

test in 4.13:
seems InstalledEngineFilters.ini has changed to InstalledEngineFilters.xml, but code is outdate.

Yep ā€“ it appears that Epic are phasing out Rocket in favor of a new build process. There are some notes to that effect in Unreal Engine 4.13 Preview - Announcements and Releases - Unreal Engine Forums (with suggested command lines for those who would like to start investigating this), and people have been investigating how to use the new build mechanism in Current advice for distributing custom engine builds to team without recompiling? - Engine Source & GitHub - Unreal Engine Forums . I am not planning to dig into this myself though, as I currently donā€™t work with UE4.

thanks Kalms!
i will read the topic and try their new methods