Announcement

Collapse
No announcement yet.

GitHub dependency zips and latest-preview snapshots

Collapse
This is a sticky topic.
X
X
  • Filter
  • Time
  • Show
Clear All
new posts

    GitHub dependency zips and latest-preview snapshots

    I’ve just updated the GitHub README with new instructions for getting UE4 dependencies - there's no need to manually download and extract zip files for the master or 4.6 preview branch; you can just run the Setup script in a clean repository, and it'll do everything for you.

    As well as downloading the dependencies matching your repository, this will register Git hooks to sync dependencies every time you checkout or merge. It'll only download the differences needed to update your working tree up to the right state, and it can be safely interrupted or resumed at any point. It won't clobber changes you make locally without prompting you first. This system works for any commit, not just for the snapshots of 'latest-preview' tagged builds that we've been limited to until now. If you want to track the master branch in real time, you can! If you want to stick to stable builds verified by our QA team you can track the new promoted branch, which receives merges from master whenever a good build is available.

    Of course, this being Epic - the source code for all this is available for you to pore over, and we'll eagerly take feedback (or pull requests) to improve on it! Some quick pointers - the tool used for downloading is in the repository under Engine/Source/Programs/GitDependencies, and the manifests of files to download are Engine/Build/Commit.gitdeps.xml and Engine/Build/Promoted.gitdeps.xml. Enjoy!

    (Note to PS4 and XboxOne developers - you'll still need to download NDA portions of the Engine from Epic's FTP site as well as using this system. New zips will be uploaded for every build in the promoted branch.)

    #2
    This is excellent stuff, thanks!
    @kylerocha
    Luminawesome Games
    http://luminawesome.com/

    Comment


      #3
      Smart addition, I just wish this system was implemented directly into the launcher to manage and handle the engine source code and stored remotely on an Epic Games server performing these tasks, I know it would require more networking resources, but working with the GitHUB client hasn't been the best experience, the GitHUB client keeps crashing on me and it's been confusing to deal with when issues come up which require troubleshooting it further.

      Thank you for this change, it should also hopefully save tons of bandwidth, I went over a 150GB bandwidth limit first time ever awhile back, when you add the source code, plus the regular installs, updates, and marketplace content, with video tutorials and more, oh plus your other third party and standard computer updates, it can quickly hit high on bandwidth usage, I just hope others are aware of that.
      Last edited by KnightTechDev; 11-14-2014, 07:45 PM.
      KT Dev Blog:
      https://sites.google.com/site/knight...evelopers-blog

      Comment


        #4
        setup.bat is ace
        well done and thanks Epic
        tegleg.co.uk - indie electronic music label
        Android + HTML5 WIP Physics Game
        PC Games - Android Apps

        Comment


          #5
          This is cool.

          Wondering if it's possible for the script to only download the libs for the platform you are on to save on bandwidth. Unfortunately I live in a country where internet isn't unlimited to 3GB+ each time I want to update libs eats into my allowance easily.

          Cheers
          Dan
          Visual Effects Artist, Weta Digital, Wellington New Zealand
          BLOG www.danielelliott.co.uk
          @danielelliott3d https://twitter.com/danielelliott3d
          Unreal Engine and VFX Tutorials https://www.youtube.com/user/DokipenTechTutorials
          2015 Showreel: https://vimeo.com/116917817

          Comment


            #6
            Originally posted by dokipen View Post
            This is cool.

            Wondering if it's possible for the script to only download the libs for the platform you are on to save on bandwidth. Unfortunately I live in a country where internet isn't unlimited to 3GB+ each time I want to update libs eats into my allowance easily.

            Cheers
            Dan
            It filters out binaries for Windows, Mac, HTML5 and Android depending on your current platform and which SDKs you have installed. It always gets Linux binaries at the moment due to the possibility of cross-compiling, but I will see if I can fix that.

            This new system doesn't require you to download 3gb every time though - it only downloads deltas from what you currently have, which should reduce your download significantly.

            Comment


              #7
              Cool stuff! So do you recommend using the promoted branch? Is the intention to use it the same way is in P4 where we checkout the labels for each release? Using the same branch makes it far better to updated and merge our own forks. I already had some issues when I merged my dev branch with a new release, so fixing it by hand was quite tedious
              Sr. Engine Programmer @ www.playspace.com - moritzwundke.com
              Remember: be polite and respect other peoples opinions - Join the Unofficial Unreal Discord Channel - Found a bug? Then use the Bug Report Form to get it fixed ^^

              Comment


                #8
                I've set up a separate rolling branch for releases (see my post here) - you may want to stick to that; it's pretty similar to the way the P4 labels work for releases. Promoted builds mainly focus on basic editor functionality, and won't be anywhere near as refined as the releases.

                Comment


                  #9
                  Originally posted by Ben Marsh View Post
                  This new system doesn't require you to download 3gb every time though - it only downloads deltas from what you currently have, which should reduce your download significantly.
                  thats good to know. if we then want to build with new source updates, do we just sync github and re-run dependency script? I have been downoaded source zips from github. so is it just the third party folder that needs copying across?
                  Visual Effects Artist, Weta Digital, Wellington New Zealand
                  BLOG www.danielelliott.co.uk
                  @danielelliott3d https://twitter.com/danielelliott3d
                  Unreal Engine and VFX Tutorials https://www.youtube.com/user/DokipenTechTutorials
                  2015 Showreel: https://vimeo.com/116917817

                  Comment


                    #10
                    If you're using Git, it will automatically download the new dependencies whenever you checkout or merge something. You can run Setup.bat at any time too.

                    With the source zips it's a bit trickier to make sure you've got a clean sync without downloading everything again - you can't easily update the source code in place without leaving deleted files around, and there's no clean way to separate out things in the zip from things in the dependency downloads. There's the content folder, samples, templates, bits of the ThirdParty folder - all mixed together within the same folders as the source.

                    Comment


                      #11
                      Found a very nasty side effect: additional files are not cached anywhere outside the working copy. As a result, after "git clean -fdx" every time quite slowly downloaded 3GB.

                      Comment


                        #12
                        Are the dependency files protected to be fetched only by active subscribers, like github access to the sources?

                        Comment


                          #13
                          Found a very nasty side effect: additional files are not cached anywhere outside the working copy. As a result, after "git clean -fdx" every time quite slowly downloaded 3GB.
                          Good point. I'll try to add caching for the 4.7 release.

                          Are the dependency files protected to be fetched only by active subscribers, like github access to the sources?
                          No, you can get dependencies for any release you already have.

                          Comment


                            #14
                            Looks good. Presumably setup.bat now downloads Required1of2 & Required2of2? Where do I get optional.zip from now for VS2012 support?

                            Comment


                              #15
                              No, it doesn't download anything in zip files any more. All the individual files you need are stored as compressed chunks of data on our servers, and it will only download the files you need. That'll exclude things like Mac libraries on Windows, or Android files if you don't have the Android SDK installed.

                              VS2012 files will be included automatically on Windows.

                              Comment

                              Working...
                              X