Announcement

Collapse
No announcement yet.

Distributing custom build to team.

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    Distributing custom build to team.

    Hi all,

    I have made some modifications to the editor and engine for internal use and I look into the best way to distribute this to the rest of my team. One thing I noticed is that when making new c++ projects based on the engine compiled from github is that it will recompile a lot of the engine source when the 'Development' configuration is used. I made one blank c++ project from the source compile from github and from the 4.7.6 build supplied by Epic through the launcher to compare the difference. The project from the official build does not recompile the engine source.

    One thing I noticed was that the projects derived from the github source build recompile a lot of the plugins into static libs with a unique name. For example, I made a bunch of test projects in the last few weeks to prototype stuff and for every one I have a .lib file located in UnrealEngine\Engine\Plugins\2D\Paper2D\Binaries\Win64 all called [ProjectName]-Paper2D-Static.lib. The same happens with all the other plugins.

    Before I distribute the modified engine to our team I would want to disable this behavior and make it behave more like the distribution from Epic. One thing I noticed is that the engine provided by Epic does not have the UE4Client/UE4Editor/UE4Game/UE4Server .Target.cs files supplied and also not the source to the programs like UnrealHeaderTool. It does have the source to the engine but not to the plugins. But I could not find any documentation of what to include and exclude.

    My question is how to make a distribution of a modified version of the engine for my team that behaves similar to the one provided by Epic. What steps need to be done and are there tools/scrips available to do this?

    Cheers,
    Pjotr
    Working at the University of Groningen
    UE4 on a huge display
    UE4 VRPN plugin

    #2
    I would be interested on this too: did you look at answer hub?

    I think the useful keyword is that you need to do a "rocket build.
    [Gamedev programmer at Darewise (Paris) - We are hiring]
    UE4 Git LFS 2.x Source Control Plugin v2.9 for UE4.20 - (v1 integrated by default in Beta status since UE4.7)
    UE4 Plastic SCM Source Control Plugin (1.3.0 for UE4.21)

    Comment


      #3
      Rocket is the code name that we give to the launcher-distributed version of UE4. As-of 4.8, you should be able to build it yourself using this command line:

      cd Engine\Build\BatchFiles
      RunUAT.bat gubp -Node=GatherRocket -CleanLocal -TargetPlatforms=Win32+Win64

      The build process is pretty convoluted, but I'd be happy to answer questions about it. Most things are included by virtue of being build products, but there are rules in Engine/Build/InstalledEngineFilters.ini that specify extra files to include.

      The thing about plugins using different prefixes for different projects is because UnrealBuildTool lets you set up a custom compile environment for monolithic targets by default. The general idea is that normally all games use the same editor binaries, but when you're compiling just your game you might want to turn on some particular compiler optimization or linker switches. You can override that behavior in your target.cs file by returning true from the ShouldUseSharedBuildEnvironment() function. We disable it for all games in Rocket, precisely because we want everyone to be able to use the same static libraries.

      "Rocket" encompasses everything particular to that distribution of the engine though, and may not be tailored to your needs. Some functionality exists to allow you to create "precompiled" builds of the engine for your own game without it being a "Rocket" build per-se: if you compile your game, appending -precompile to the UnrealBuildTool tool command line, it will create static libraries for any modules you list in the GetModulesToPrecompile() function of your target rules. You can run GenerateProjectFiles.bat with the -useprecompiled flag to have it treat everything you've precompiled as read-only.

      Comment


        #4
        That sounds pretty good. Thank you for the write up. Looking forward to 4.8.
        Working at the University of Groningen
        UE4 on a huge display
        UE4 VRPN plugin

        Comment


          #5
          Originally posted by Ben Marsh View Post
          Rocket is the code name that we give to the launcher-distributed version of UE4. As-of 4.8, you should be able to build it yourself using this command line:

          cd Engine\Build\BatchFiles
          RunUAT.bat gubp -Node=GatherRocket -CleanLocal -TargetPlatforms=Win32+Win64

          The build process is pretty convoluted, but I'd be happy to answer questions about it. Most things are included by virtue of being build products, but there are rules in Engine/Build/InstalledEngineFilters.ini that specify extra files to include.

          The thing about plugins using different prefixes for different projects is because UnrealBuildTool lets you set up a custom compile environment for monolithic targets by default. The general idea is that normally all games use the same editor binaries, but when you're compiling just your game you might want to turn on some particular compiler optimization or linker switches. You can override that behavior in your target.cs file by returning true from the ShouldUseSharedBuildEnvironment() function. We disable it for all games in Rocket, precisely because we want everyone to be able to use the same static libraries.

          "Rocket" encompasses everything particular to that distribution of the engine though, and may not be tailored to your needs. Some functionality exists to allow you to create "precompiled" builds of the engine for your own game without it being a "Rocket" build per-se: if you compile your game, appending -precompile to the UnrealBuildTool tool command line, it will create static libraries for any modules you list in the GetModulesToPrecompile() function of your target rules. You can run GenerateProjectFiles.bat with the -useprecompiled flag to have it treat everything you've precompiled as read-only.
          Hi, Ben!

          Thank you for this information, it is very valuable!

          I have a few questions:

          1. Is there any way to recompile certain engine modules without recompiling the rest of them? Provided that UBT is launched with -useprecompiled option and all engine static libraries exist. I keep .dll and .lib files under source control but I don't keep .pdb and .obj files there. I want to be able to quickly make changes to private parts of certain modules and rebuild only these modules. This would allow faster iteration on engine changes.

          2. When using -useprecompiled option, UBT still rebuilds plugins (obviously because they are not precompiled). Is there any way to tell UBT to not try to build plugins?

          If this is not currently possible, what kind of modification can be made to UBT to implement these behaviors?
          Realistic Atmosphere 2.0 | YouTube | Forum | Marketplace

          Comment


            #6
            You can use the -module XXX argument with UBT to just compile a specific module (or set of modules, if it's specified more than once). It's not integrated into Visual Studio or anything like that though, I'm afraid.

            The precompiling of plugins is hard-coded in SetupPlugins() in UEBuildTarget.cs (the code under the comment 'Set the list of plugins that should be built').

            Comment


              #7
              @Ben Marsh
              Thank you! It worked!

              Here's the fix I made to UBT, I post it here in case it is useful for the main engine:

              Code:
              			// Set the list of plugins that should be built
              			if (bUsePrecompiled)
              			{
              				// Only build project plugins
              				BuildPlugins = new List<PluginInfo>();
              				foreach (PluginInfo Plugin in EnabledPlugins)
              				{
              					if (Plugin.LoadedFrom == PluginLoadedFrom.GameProject)
              					{
              						BuildPlugins.Add(Plugin);
              					}
              				}
              			}
              			else
              			{
              				if (bPrecompile && TargetType != TargetRules.TargetType.Program)
              				{
              					BuildPlugins = new List<PluginInfo>(ValidPlugins);
              				}
              				else
              				{
              					BuildPlugins = new List<PluginInfo>(EnabledPlugins);
              				}
              			}
              Last edited by Robert Khalikov; 08-17-2015, 08:15 PM.
              Realistic Atmosphere 2.0 | YouTube | Forum | Marketplace

              Comment


                #8
                @Ben Marsh - I get an error when I run the command. Any idea what is wrong?

                Excerpt of the output that includes the error:

                ScriptCompiler.LoadPreCompiledScriptAssemblies: Found 9 script DLL(s).
                GUBP.ExecuteBuild: ************************* GUBP
                ProjectUtils.CompileAndLoadTargetsAssembly: Compiling targets DLL: G:\windows-slave-1\windows-build\build\UnrealEngine4\Engine\Programs\AutomationTool\Saved\Rules\UATRules_BaseEngine_.dll
                ProjectUtils.CompileAndLoadTargetsAssembly: Compiling targets DLL: G:\windows-slave-1\windows-build\build\UnrealEngine4\Engine\Programs\AutomationTool\Saved\Rules\UATRules1901819006.dll
                BranchInfo..ctor: Base Engine:
                BranchInfo..ctor: 1 Code projects:
                BranchInfo..ctor: 1 Non-Code projects:
                BuildCommand.Execute: ERROR: BUILD FAILED
                Program.Main: ERROR: AutomationTool terminated with exception:
                Program.Main: ERROR: Exception in mscorlib: The given key was not present in the dictionary.
                Full output: http://pastebin.com/NNLKgpNU
                Descent: Underground |

                Comment


                  #9
                  FWIW it's not working here either. Getting "ERROR: Exception in mscorlib: The given key was not present in the dictionary." as above. I am seeing this in the very latest release branch (4.9.1), and also on 4.8.3

                  Full log: https://gist.github.com/TTimo/410915fe1fee2f15b6b2

                  Comment


                    #10
                    Originally posted by TTimo View Post
                    FWIW it's not working here either. Getting "ERROR: Exception in mscorlib: The given key was not present in the dictionary." as above. I am seeing this in the very latest release branch (4.9.1), and also on 4.8.3

                    Full log: https://gist.github.com/TTimo/410915fe1fee2f15b6b2
                    I have the same issue. "ERROR: Exception in mscorlib: The given key was not present in the dictionary." Has anyone got a fix or workaround? I'm using the following command line: RunUAT.bat gubp -Node=GatherRocket -CleanLocal -TargetPlatforms=Win32+Win64

                    Comment


                      #11
                      Likewise, same issue. Anyone found a solution yet?

                      Comment


                        #12
                        Hi there,

                        I've recently built out a custom version of UE4 for my company to use as we needed to make some changes to the renderer.
                        After building out the engine and testing it on my own machine, I am now ready to roll it out to the rest of the team.

                        One thing I have noticed is that the size of the engine version is around 42GB compared to the standard version at 11GB.

                        A lot of this extra data is held in Engine/Intermediate/Build/Win64.

                        The standard engine has two folders in here "UE4" and "UE4 Editor" where as my custom version has these plus "Inc" "UE4Game" and "UnrealHeaderTool".

                        Are there any console commands I can run that will clean out all the extra stuff and just leave me with a smaller build I can distribute to my team.

                        Thanks!
                        Last edited by AgentMilkshake1; 09-29-2015, 05:57 AM. Reason: typo

                        Comment


                          #13
                          You can take out the following three folders when copying/distributing to the team:
                          Code:
                          .git
                          Engine/Intermediate
                          Engine/DerivedDataCache
                          Cheers,
                          • Follow me on twitter
                          • Visit our website traverse.world
                          • Checkout our game's forum thread

                          Comment


                            #14
                            Fantastic, I'll give it a go!

                            Cheers

                            Comment


                              #15
                              I too have that same error as Stormwind (Program.Main: ERROR: Exception in mscorlib: The given key was not present in the dictionary.)
                              Any ide how to fix this ?

                              Comment

                              Working...
                              X