[TUTORIAL] How to use Setup.bat and how to reduce download size

Development Programming & Scripting C++
, , ,
1

Hi guys, here’s some useful information for those who care about download size and/or disk space taken by the engine dependencies.

By default, Setup.bat always downloads files for your current platform: Win32 and Win64 if you are on Windows and Mac if you are on Mac. But you will also get dependency files for iOS and Linux even if you won’t need them. It also downloads HLML5 dependencies if you have Emscripten installed and Android dependencies if you have NDK.

This is quite clever but what if you want to decide which files to download? You can use two command-line options: -exclude and -include.

Use -exclude to specify platforms you are not going to use. For example, if you don’t want Linux and iOS dependencies run this:


Setup.bat -exclude=Linux -exclude=IOS

Use -include to specify what you want to download. For example, you don’t have NDK and Emscripten installed (yet) but want to get Android and HTML5 files anyway:


Setup.bat -include=HTML5 -include=Android

Here is the list of platforms to use with these two options: Win32, Win64, Mac, Linux, Android, IOS, HTML5.

Useful, isn’t it? But this is not all! By default, when you are on Windows the script downloads library files for both VS2012 and VS2013, what if you are only going to use one version and won’t need files for the other? Just add an -exclude option:


Setup.bat -exclude=VS2012

or


Setup.bat -exclude=VS2013

In fact, this option simply works as a filter that will filter out all the files that contain specified folder in their paths, since all version-dependent Visual Studio libraries are located in folders “VS2012” and “VS2013” they will be filtered out by this option! This can greatly reduce your disk space usage!

Well, just wanted to share this for those who didn’t figure that out yet. :slight_smile:
Hope this information is useful!

Thanks for reading!

Robert.

P.S. Thanks for the sources of the GitDependencies program, Epic! This is how I learned all this stuff!

6 Likes
2

Thanks, that’s actually very helpful! :slight_smile:

4

There are some other folders that can be filtered out. If you are not targeting Windows RT you can add WinRT to the list of excluded directories. Also, I noticed that some libraries store their files in non-standard folders, examples are Linux32, osx64, x64 (instead of Win64). So if you are not targeting these platforms and want to reduce the size even more, you can add these folders too.

5

Thanks for this. Got me back about 5GB of space.

6

If I exclude the VS2012 files I get the error “LNK1181: cannot open input file 'silk_common.lib” with UE4.6.1

7

Same for me with UE4.7: -exclude=VS2012 give a link error, but also also -exclude=Win32


20>  [1/179] Link UE4Editor-Engine.dll
20>LINK : fatal error LNK1181: cannot open input file 'ThirdParty/libOpus/opus-1.1/win32/VS2012/x64/Release/\silk_common.lib'
20>  -------- End Detailed Actions Stats -----------------------------------------------------------
20>ERROR : UBT error : Failed to produce item: C:\Users\srombauts\workspace\UE47\Engine\Binaries\Win64\UE4Editor-Engine.dll

This is because silk_common.lib is only available under the path ‘ThirdParty/libOpus/opus-1.1/win32/VS2012/x64/Release/\silk_common.lib’

Currently using:


Setup.bat -exclude=WinRT -exclude=Linux -exclude=Linux32 -exclude=osx64 -exclude=IOS
8

Presumably Mr. Khalikov posted this guide without actually testing it himself.

9

Amazing!

I’m used:


Setup.bat -exclude=Linux -exclude=Linux32 -exclude=osx32 -exclude=osx64 -exclude=IOS
10

Very helpful and informative thread, Thanks.

Currently using:


Setup.bat -exclude=WinRT -exclude=Linux -exclude=Linux32 -exclude=osx64 -exclude=osx32 -exclude=IOS -include=HTML5 -include=Android
11

i’m just confused? sorry I don’t know anything about programming but please somebody could explain where should I include the -include or -exclude option in setup.bat? do I have to include it inside the setup.bat? ei…

@
setlocal
pushd %~dp0

rem Sync the dependencies…
.\Engine\Binaries\DotNET\GitDependencies.exe --prompt %*
if ERRORLEVEL 1 goto error

rem Setup the git hooks…
if not exist .git\hooks goto no_git_hooks_directory
Registering git hooks…
echo #!/bin/sh >.git\hooks\post-checkout
Engine/Binaries/DotNET/GitDependencies.exe >>.git\hooks\post-checkout
echo #!/bin/sh >.git\hooks\post-merge
Engine/Binaries/DotNET/GitDependencies.exe >>.git\hooks\post-merge
:no_git_hooks_directory

or shoud I rename the setup.bat and include this options?
please i’m a beginner know nothing about programming. :frowning:

12

You don’t include them in the batch file. You pass them as arguments when you run them from command line (On Windows, it’s CMD)

You run CMD, cd to the folder (cd C:<some path>\UnrealEngine4), then do “Setup.bat -include=Android -exclude=WinRT” etc.

13

thanks for clearing up :slight_smile:

14

or, open up notepad, paste in this “Setup.bat -exclude=WinRT -exclude=Linux -exclude=Linux32 -exclude=osx64 -exclude=IOS -exclude=HTML5” that will excludes what you see typed, you can add or take away from there, but I use that one for my PC and for Android.

save the file as a .bat in the same folder as the Setup.bat

15

these commands still work for version 4.10.4 !! i just want to work on win7x64
Right now installing ue4 for learning purpose so doesn’t need all the dependencies.

can i use this “Setup.bat -exclude=WinRT -exclude=Linux -exclude=Linux32 -exclude=osx64 -exclude=IOS -exclude=HTML5”
what is the size of these dependencies ?

Tested :slight_smile:
its also works on 4.10.4 :slight_smile:
dependencies size around 3Gb & Full dependencies around 3.5 Gb but note !!
when i download 3gb dependencies the size of the unreal folder gone to around 9Gb & after building engine its 30Gb wow :0

so adding full dependencies & building the whole engine require more space & time !!

16

By the way, there’s now a more flexible way to customize dependency download. Should be included in 4.11.

17

ok Sir :slight_smile: …

18

I like the whole concept but then there is a chaotic situation if you want to build for many platforms. My recent experience with Linux building for example was a pain in the butt. TLDR you have to redownload dependencies,rebuild the engine,the editor , some extra modules. Stuff may not work from the first time etc.

Let alone that, the whole idea reminds me of some people spamming the facebook group in search of ways to save a ton of bandwidth while downloading the engine from github or the launcher,or trying to build the engine with a low-end system.

Anyway, It’s a no-go for me. I will stick with grabbing all the dependencies (even linux ones).

19

Just want to add to this, using Exclude VS2013 and VS2012 will cause your builds to fail. Tested recently in 4.15.

See findings here:

20

A few updates for this thread. I tried a few times with a 4.15 build and this is what I found.

  • You can’t exclude ‘IOS’ platform, even if you’re not using it. If you do, the iPhonePackager module will be missing a few icons and won’t be able to build, preventing the engine from building at all.
  • If you want to exclude ‘Switch’ dependencies, you can add

-exclude=Dingo
  • You can’t seem to use -exclude=VS2012 or -exclude=VS2013, because there are dependencies in those folders which don’t exist in VS2015 folder, and are required.

If you want to decrease your download size, you can also add a .gitdepsignore file to the root folder of the repository, and add the following lines:



/Engine/Documentation/**
/Engine/Extras/**
/FeaturePacks/**
/Samples/**
/Templates/**
!/Engine/Extras/Redist/**
!/Engine/Extras/VisualStudioDebugging/**

This removes a lot of the content download, but keeps the things that are still required for the engine to run.

1 Like
21

Hey guys, just wanted to chime in. gitdepsignore and exclude= is super useful. I didn’t find it specified exactly anywhere, but .gitdepsignore has to be in the <workspace>/UE4/ folder. I’m only having my programmers use Setup.bat. For artists, I’ve written a python script for having TeamCity automatically update and submit Editor/Game binaries on every commit, and include all Content folders as well in the case of an engine upgrade.

Here’s my .gitdepsignore:


/Engine/Documentation/**
/Engine/Extras/**
!/Engine/Extras/Redist/**
!/Engine/Extras/VisualStudioDebugging/**
!/Engine/Extras/VisualStudioSnippets/**
/FeaturePacks/**
/Samples/**
/Templates/**
/Engine/**/Android*/**
!/Engine/Plugins/Runtime/AndroidPermission/**
/Engine/**/IOS*/**
/Engine/**/TVOS*/**
/Engine/**/Google*/**
/Engine/**/LeapMotion/**
/Engine/Plugins/Online/OnlineSubsystemFacebook/**
/Engine/Plugins/Online/OnlineSubsystemAmazon/**
/Engine/Plugins/Online/OnlineSubsystemGoogle/**
#i'm excluding engine and plugin content because we're including it in Perforce
#including a / after Engine breaks this because it evaluates to Engine/.*/Content which will not find Engine/Content
/Engine**/Content/**

Here’s part of my Setup.bat:


set EXCLUDES= -exclude=Linux -exclude=Mac -exclude=Android -exclude=TVOS -exclude=IOS -exclude=GoogleVR -exclude=GoogleTest -exclude=LeapMotion -exclude=Dingo -exclude=Switch

rem Sync the dependencies...
.\Engine\Binaries\DotNET\GitDependencies.exe %PROMPT_ARGUMENT% %* %EXCLUDES%

Thanks for this thread!

EDIT: I should note that some of the things I excluded do require engine modifications. I’ve basically removed all Android, IOS, TVOS references in the engine, and deleted related projects.

1 Like