What is the recommended workflow for storing and updating 3rd party libraries and SDKs in relation to the engine source?
We’ve tried a brute-force approach of stuffing them all together in a single git branch, so that a programmer can check it out and build the engine immediately. Unfortunately, git started having problems with large library files (200MB or more) that are sometimes part of 3rd party SDKs. A possible workaround is using an x64 git build, but before I try that, I’d like to ask the community, as well as Epic Games, about their experience.
Git doesn’t handle large binary files well - every commit is strongly hashed from its contents, which includes a reference to its previous commit as well as all the files in the tree, so you can never remove those large files once they’re in the repository. Your large repository will just keep on getting larger, and fixing it by rewriting the repository history will be hugely disruptive to your team.
We’ve made this mistake with Unreal Tournament, and it’s incredibly slow to clone the now-9gb repository. In contrast, a clean repository with just the UT source code is something like 20mb. We’re embarking on a mission to transition at the moment, but it’s likely to be painful.
