Announcement

Collapse
No announcement yet.

Compiling game modules with github source to be used by non github source developers

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Compiling game modules with github source to be used by non github source developers

    I am wondering about a particular configuration detail in the github source, but I will describe why first to give an accurate picture of the driving reasons.

    Our team operates in such a way that only a small subset of the team actually needs to do programming, and those developers are the only ones who set up a coding environment. Whenever programmers commit changes (myself included) we check the game's module DLLs into source control so that the rest of the art team gets the changes.

    Within the programmers there are two sensible configurations. Either compiling from the headers and libraries included in the distributed editor, or compiling from the full github source.

    Currently I program using the full github source for the power this provides in debugging the engine code. It is important to note that I do not do this to make any edits to the engine, which I really don't need to do at this time and likely for the distant future. Everything seems to go great, save for one issue I have run into since the release of 4.4.

    I have noticed that when I commit game module DLLs compiled from the full github source, my artists let me know they were getting the following error when they try and open the game project with the downloaded editor:

    "Project modules are missing or out of date. Would you like to recompile them?"

    Upon looking at the game's log created from this startup the following message is there:

    UE4Editor-<MyGame>.dll (API version 0), but it was incompatible with the current engine API version (2255576).

    I was able to reproduce this by downloading 4.4 of the editor and saw the same issue.

    Upon examining the source, one can see this error is triggered by the failure of the following check:

    Code:
    bool FModuleManager::CheckModuleCompatibility(const TCHAR* Filename)
    {
    	int32 ModuleApiVersion = FPlatformProcess::GetDllApiVersion(Filename);
    	if(ModuleApiVersion != MODULE_API_VERSION)
    	{
    		UE_LOG(LogModuleManager, Warning, TEXT("Found module file %s (API version %d), but it was incompatible with the current engine API version (%d). This is likely a stale module that must be recompiled."), Filename, ModuleApiVersion, MODULE_API_VERSION);
    		return false;
    	}
    	return true;
    }
    From what I can see, MODULE_API_VERSION is a preprocessor definition that derives from the BUILT_FROM_CHANGELIST definition inside of "Engine\Source\Runtime\Launch\Resources\Version.h".

    This definition was set to the following values during the building of the distributed editor:
    Code:
    #define ENGINE_VERSION 2255576
    #define ENGINE_VERSION_HIWORD 34
    #define ENGINE_VERSION_LOWORD 27352
    
    #define BUILT_FROM_CHANGELIST 2255576
    #define BRANCH_NAME "++depot+UE4-Releases+4.4"
    But these are zeroed out in source code and thus the github distribution of the 4.4 release branch:
    Code:
    #define ENGINE_VERSION 0
    #define ENGINE_VERSION_HIWORD 0
    #define ENGINE_VERSION_LOWORD 0
    
    #define BUILT_FROM_CHANGELIST 0
    #define BRANCH_NAME "UE4"
    I tested this out and sure enough after changing the value I was able to compile DLL's that our artists could load and all was good and right in the world.

    There are two big things I'm wondering:

    First, why did this check not prevent loading the game project past, but suddenly start after 4.4.

    The other, if this is in fact a permanent check in the source, is if there would be a way that developers who are compiling the engine from source could indicate that they have not made changes to the source, and target developers using the downloaded editor (without having to go in and edit the source that is).

    The big thing I am wondering is why in the github tags for releases, the changelist # from perforce is not inserted into this source file in the perforce submission that creates the tag in git (not sure if I am phrasing that correctly, but something to that effect). To me at least, it seems it would make sense for the tagged github source releases to have the state of this Version.h file reflect the public releases of the editor instead of baking those changelist #'s in during the release process.

    This would then allow developers who haven't made any edits to the engine source to build module DLLs that are by default compatible with artist's downloaded editors. I know that it would be easy enough for me or others with this process to simply edit the changelist # into the engine source's Version.h after downloading, but I also feel that this would be something that is easy to forget to do, and then potentially lead to the submission of broken DLLs if this is not caught before the next submit.

    For developers who have fully branched the engine for their own studio and keep the full source in version control, it would be easy enough to simply completely change these #'s, or even the way the module checking happens. For those developers who actively develop for the engine through github, but for whatever reason want to preserve that they have a custom version whose changelist # shouldn't be compatible with the latest release, the master branch in github could simply keep the zeroed out values in the Version.h it has now. That is internally in Epic these values in the depot would keep the zeroed values, and they would only be changed upon tagging a public editor / github source release.

    Either way I'm just doing a lot of speculating and I would love if an engine developer could explain why things are the way they are. Either way it would be nice to have a way to solve the above situation so that developers using the github source releases may compile modules for use by non github users. This is both my own sake, and perhaps for anyone else who might potentially run into this issue in the future and not know the cause.
    Last edited by Merlin; 08-20-2014, 11:58 AM.

    #2
    Good thread, I completely agree. I have exactly the same problem and would also like to have some information from a developer.
    Also there seem to be some implications coming from BUILT_FROM_CHANGELIST regarding licensee status and possibly other things related to analytics and crash reporting.

    Comment


      #3
      Shameless bump .

      Still hoping to get info on this if possible?

      In particular I'm wondering if its possible to get either Version.h, or if not that ModuleVersion.h to have BUILT_FROM_CHANGELIST and/or MODULE_API_VERSION to match that of editor releases for the corresponding tagged github source releases.

      Comment


        #4
        You're spot on.

        The general assumption is that if you're compiling the engine yourself, there's a good chance you're looking to change it. We use the zero values to distinguish a local build (where you may have local changes that nobody else has) against a canonical build (which is exactly the same for a bunch of people). Loading mismatched DLLs can just throw up obscure Windows loader errors at best (in the cases that a function signature has changed), and cause subtle data corruption at worst (if a class layout has changed). You can update those defines to match if you're sure your build is binary compatible with the launcher build, but it's a much safer default to assume that it's not.

        Those version numbers are used for a few other things too, but probably the most important is package versioning. The way that classes are serialized into .uasset and .umap packages is very tolerant to properties being added and removed, but we want to make sure that an artist on an old build of the editor doesn't accidentally load and save a package while silently discarding any class properties that didn't yet exist in that build. We bake the version number into any saved packages and refuse to load any package that has a newer version than the editor.

        Comment

        Working...
        X