Github and large files

Development Programming & Scripting C++
,
1

Hi guys!

I’m about to tear my head apart atm.
Yesterday when I were going to start me and my friends project, Unreal told me that I had to recompile the Source for some reason.
Which is weird since nothing had been done to the project since the last time I used it. My friend could still open the project, and we were both using the same version of the engine.
The solutions I found on the Internet didn’t work.
I didn’t know what to do, but downloading Visual Studio and recompiling it seemed to do it.

After that I made some changes and wanted to commit, but the files became too large. Guessing it had to do with the major changes of the compilation.
Now I´ve started a new repository, to start over on this version I´ve got. But still I can’t commit since the project is too “big”.
Which is weird since the project is around the same stage as it were when we started using Git/(Sourcetree), so the size shouldn’t have changed that much.

If the project is around 1.5GB how do you then make a commit/start a new repository?
It even tells me that 100mb is the maximum amount you can commit.

2

Afaik, git has issues with binary and/or large files. You probably want to use SVN or Perforce.

But i think a “.gitignore” file will solve your issue.

Create a file near your .uproject file called .gitignore and add these lines:



*.slo
*.sln
*.lo
*.o
*.obj
*.gch
*.pch
*.so
*.dylib
*.dll
*.mod
*.lai
*.la
*.a
*.lib
*.exe
*.out
*.app
*.log
*.pdb
Intermediate/*
Saved/*

Also Github has a 100MB / file limit.

3

While Bitbucket doesn’t have this limit, their policy is to be reasonable about what you upload. Essentially Wheaton’s Law. (Don’t be a jerk*)

I’ve uploaded files significantly larger than 100MB on their site, but I was very careful to not change them often/at all. And an Unreal project is definitely not reasonable for git (any host) without culling out all binary files first. Note that’s the reason Epic have a set of zips included with their Github repository, and you have to download all the binary files separately. Internally (I believe) they use Perforce and then some process pulls out the binaries and automatically mirrors it on Github.

*Paraphrased for word filter.

4

Github has prebuilt gitignore files for unreal engine, here’s the one that was built for me:
(p.s. these comments are formatted in markdown, and those headers are actually hashtag comments)

Visual Studio 2015 user specific files

.vs/

Compiled Object files

*.slo

*.lo

*.o

*.obj

Precompiled Headers

*.gch

*.pch

Compiled Dynamic libraries

*.so

*.dylib

*.dll

Fortran module files

*.mod

Compiled Static libraries

*.lai

*.la

*.a

*.lib

Executables

*.exe

*.out

*.app

*.ipa

These project files can be generated by the engine

*.xcodeproj

*.xcworkspace

*.sln

*.suo

*.opensdf

*.sdf

*.VC.db

*.VC.opendb

Precompiled Assets

SourceArt/**/*.png

SourceArt/**/*.tga

Binary Files

Binaries/*

Plugins//Binaries/

Builds

Build/*

Whitelist PakBlacklist-.txt files

!Build/*/

Build/*/**

!Build//PakBlacklist.txt

Don’t ignore icon files in Build

!Build/**/*.ico

Built data for maps

*_BuiltData.uasset

Configuration files generated by the Editor

Saved/*

Compiled source files for the engine to use

Intermediate/*

Plugins//Intermediate/

Cache files for the editor to use

DerivedDataCache/*

 

 

 

 

However, I’m still getting an exceeded file limit, which is kind of annoying. The file is a smart object collection ipch file, is it okay if I put that in the gitignore as well?

Error message:

The push operation includes a file which exceeds GitHub’s file size restriction of 100MB. Please remove the file from history and try again.

File causing error:

SMARTOBJECTCOLLECTION.ipch (653.31 MB)

1 Like
5

Yes, I am new to Games programming and Unreal but I am absolutely horrified by the size of the commits.
Is there a reasonable way to limit this?
The size of the files commited to source control is way too big
Is there any plan to address this? Or a way to be more agressive on the .gitignore?

6

i probably wouldn’t commit the precompiled header files, pch, ipch … those are something the editor will generate on it’s own when it needs them. the whole .vs directory probably shouldn’t be included in the source control, and/or your visual studio is probably better off if you have all PCH generation shoved off to another disk (i just did this, and VS and Unreal are much much faster, now that i have PCHs on one ssd, DDC on another, and all of Unreal on a third)

your biggest commits are going to be anything that changes in your content directory, or if you do a complete engine rebuild. Otherwise, you shouldn’t really have too much going on on a day to day basis, your game binaries, some DLLs, code, and content.