Workflow: Git & Big Assets

Development Programming & Scripting
1

The Problem

I am currently in the start phase of a new game-project with a small team. I would prefer to use git. I would be happy with it, if it weren’t for the big-file problem.

My question is how to manage the assets? Have you some experience with git and assets you want to share?

Ideas for Solutions

This are the solution I came up with (ordered from most to least favourite solutions):

1. Ignore

Just ignore the issue, until it becomes problematic. Afterwards, we still can remove the asset directories using git filter-branch and think of a different solution.

    • Full support of the UE4 Editorgit plugin and other git gui tools (like atlassian sourcetree)
    • No overhead in the workflow

2. Use some extra tools

git-fat or git-media (as suggested by the Unreal Wiki) the looks somehow promising.

3. Use Submodules with depth 1

Like svn, only the current version of assets would be stored locally.

    • When neededm the whole asset history is still avialable
    • Not easy to use with Git GUI tools like Atlassian SourceTree
  • ? Is this supported by the git plugin for UE4?

4. Use Subversion only for the asset directrories

somehow similar to the second option of this post

    • Git for C++ Code and svn for assets
    • Is the UE4 Editor git plugin able to handle this?

5. Use Subversion instead of git

    • full UE4 Editor support
    • Not able to use version controll features offline (like when coding in C++ in a train)

6. No Version Controll for assets

My least favorite solution. Synchronize using a cloud service, like dropbox or owncloud

    • No support of the UE4 Editor Source Controll plugin
    • No support for UE4 Editor blueprint merging
    • No support of debugging helpers like git-bisect
2

Hi, just a first answer about the Git plugin, about each of your points

  1. This is the normal workflow, the only one supported by the Git plugin (whom I authored). Will work flawlessly up to a certain point where your history will slow down any new git clone. In a local network and with a small team this might not be a problem. But IT IS for projects with many peoples needing to synchronize across Internet (and with many new people wanting to clone the whole repository often).
  2. git-fat, git-media and git-annex could (should?) work with the git plugin (as the new github LFS), but I never tried
    any of them as 1) always served my needs. Would love a feedback though!
  3. submodules are not supported by the git plugin for now, but is would be a good improvement to think about in the future.
  4. subversion is not supported by the git plugin
  5. subversion is not supported by the git plugin
  6. no source control is not supported by the git plugin :wink:

Also, you should certainly differentiate Blueprints to all other assets, as only Blueprints are Visually Diff-able.

3

As a Team, we’ve decided for option 6. We chose to use the BP_ prefix for blueprints, so our .gitignore script can keep blueprints in version control.

  1. After modelling a single character by sculpting it, the repository got already about 400MB large. In a LAN environment, that’s not an issue, but as I have a slow internet connection, things might become unpleasingly very fast.
    Also, I would have to make sure that git doesn’t internally pack the diffs into packages with a size larger than 100MB, as gitlab is rejecting the git push command otherwise. This would require the whole team to start fiddling around with internal git settings.
    Again, this is only a Problem, if you don’t have an own (local) git server.
  2. I’ve experimented a little with git-fat. It was working flawlessly from the terminal. When using the terminal, you didn’t even had to know, that you are using git-fat (thanks to aliases). But when using a gaphical user interface for git, this was no longer the case. One had to take special care for every commit not to mix things up.
  3. I found no way to set this up using a graphical user interface. Also when pushing changes, one has to take care of the same problems as in 1.
  4. Question of preference, we chose 6 instead
  5. SVN never really was an option for our team :stuck_out_tongue:
  6. We decided to sync the assets using a cloud based service and the source code and blueprints with git. For asset creation, every artist can use locally a version controll system of choice if he wants to. This way the whole repository won’t get blown up.
1 Like