Adding Third Party Libraries to Unreal...understanding how this works

Development Programming & Scripting C++
,
1

I’m not totally sure this is exactly the right way to approach the problem or not, but when you understand, really understand, how something works it makes it easier to produce quality content.

I really wanted to understand how the Third Party Library template works. First step is to open/create a game. And it needs to be a C++ game not a Blueprint game. Apparently you can’t see the advance choices from the green “New Plugin” button if you created the original game as a blueprint. In a new C++ game, the “New Plugin” button displays:

  • Blank
  • Content Only
  • Blueprint Library
  • Editor Toolbar Button
  • Editor Standalone Window
  • Editor Mode
  • Third Party Library <— use this choice for the quick test

Create a new Third Party Library plugin. Lets call it something fancy, “abc”. The plugin gets built “Generating Code” and you will see a popup window “Third Party Plugin” “Loaded ExampleLibrary.dll from Third Party Plugin sample.” That tells you the plug was installed, built and successfully run. You can analyse the code contained within the Plugin folder via Visual Studio. Check and understand each and every file contained within, particularly the C# files: “abc.Build.cs” and “abcLibrary.Build.cs”. Note carefully the file structure…



Plugins
    abc
        -Binaries
            ThirdParty
                abcLibrary
                    Win64
            Win64
        -Content
        -Intermediate (contains Build --> Win64 --> UE4Editor-->Development-->abc folders)
        -Resources
        -Source
            abc
                Private
                Public
            ThirdParty
                abcLibrary
                    ExampleLibrary (with folder ExampleLibrary.xcodeproj)
                    ExampleLibrary.xcodeworkspace
                    Mac
                        Release
                    x64
                        Release

What happens here is that the original content of the plugin contains that which is held within Source, Resources and Content folders. The other root folders, Binaries and Intermediate are generated folders, propagated when you add the plugin (remember the “Generating Code” popup?) And when you really dissect the code, the magic is that the abcLibrary .dll / .lib files are copied from Source into Binaries/ThirdParty folder as appropriate.

At this point we haven’t done a thing. Push a button, create a new plugin, give it a name, and look closely at stuff. Now, lets do a little test. Close the game engine, close visual studio. Delete the entire Binaries folder and delete the entire Intermediate folder. Open up Visual Studio for the game project, then rebuild. What should happen is those two folders (Binaries and Intermediate) should rebuild. But here’s the rub. That doesn’t work. The Binaries/ThirdParty folder is gone forever. And when that’s gone, the plugin fails When you open the game you will see a white box popup “Message” “Failed to load example third party library”.

I’m not getting this. Why is the original build (after the Green add New Plugin button is pushed) so much different than the Visual Studio / UE4 engine rebuild process? What this means in real life is if you are trying to introduce a real third party plugin with contained .lib/.dll precompiled libraries, they will NEVER propagate over to the Binaries folder.

Note, in the abcLibrary.Build.cs file I do see the line


// Ensure that the DLL is staged along with the executable
RuntimeDependencies.Add("$(PluginDir)/Binaries/ThirdParty/abcLibrary/Win64/ExampleLibrary.dll");

I would think that should do it? And note, I do see the origin of that code here.

Am I doing this wrong? Can anybody explain what’s happening here?

3

Interesting… Plugin rebuild process should be improved or at least documented - Plugins - Epic Developer Community Forums

4

I’ve been spending a whole lot of time trying to figure out how to let the Unreal Engine generate those library binary files. I’m assuming that this is something to do with the C# build process and that UnrealBuildTool.exe is at the root of this. From this unrealengine.com reference:

I can see the BuildConfiguration.cs file at C:\Program Files\Epic Games\UE_4.26\Engine\Source\Programs\UnrealBuildTool\Configuration

And from this reference I understand the best way to call the build tool is to use an existing batch file. As I check my computer there are a whole lot of batch files.

From location: C:\Program Files\Epic Games\UE_4.26\Engine\Build\BatchFiles



Build.bat
Clean.bat
GetMSBuildPath.bat
Linux/
Mac/
MakeAndInstallSSHKey.bat
Rebuild.bat
RunUAT.bat
RunUAT.command
RunUAT.sh

This makes sense. Now I understand what is happening when I’m making a Build call from within Visual Studio. But here’s the issue. Because I’m unable to use Visual Studio to build the Third Party Plugin Binaries folder, I suspect something else is needed. What’s not clear is how the “Add New Plugin” button works. Its calling something and that program is correctly building the Third Party Plugin example. I’m wondering what configuration parameters its using in its call.

I did add another ThirdParty Plugin, and kept close track of the build log within Unreal Editor tool I recreated that build via a batch file, with the same parameter calls as displayed in the editor, total fail. The system will not build the Binaries ThirdParty Library folder.

Am I the only person to need this stuff? How is everybody else creating functional Third Party Library tooling? (Note, I’m at Unreal 4.26)

Whoa. I just made yet another discovery. All this time I’ve been thinking that the Binaries/ThirdParty folder is created when you push the Add New Plugin button. Absolutely not true. I found the original source folder in Github for the green button new additional plugins. The path is: UnrealEngine/Engine/Plugins/Editor/PluginBrowser/Templates/

I was able to deduce from that where in my Unreal Engine install the same stuff was located. I found that at C:\Program Files\Epic Games\UE_4.26\Engine\Plugins\Editor\PluginBrowser\Templates\ , but my surprise was that the *binaries/ThirdParty *folder was there and totally populated with library content.

I had always thought that stuff was generated during the plugin installation, but I can see that it must have happened at unreal engine download and installation on my computer. That totally changes the conversation, but doesn’t explain how it was populated. The *binaries *folder is not in the github repo, so its created somehow. I will say, with this information, I don’t feel like I’m cheating if I create my own third party plugins with binaries folders pre-populated, too. Its a hack, but only half a hack. (Does that count?)

With that said, what’s the point of even having a binaries folder?

1 Like
5

ThirdParty folder usually shouldn’t be inside /Binaries at all, you keep it in same root as Binaries folder is, then customize your /Config ini files to include that folder, and Build.cs to look for libs in there for “PublicDelayLoadDLLs”.

6

Not sure we’re in the same place. **What you just said there doesn’t match with what I see in my Unreal installation. ** What do you see on your computer at *C:\Program Files\Epic Games\UE_4.26\Engine\Plugins\Editor\PluginBrowser\Templates\ThirdPartyLibrary\Binaries ? * (Obviously at the appropriate location for your drive and version…)

You are way ahead of me here on the whole static vs dynamic library implementation thing. But you are right, only the dynamic (.dll or.dylib) version of the library got pushed to the Binary / PLUGIN_NAMELibrary folder. Again, I’m just trying to understand exactly how the whole “Third Party Plugin” example, native from Unreal, works. And that example absolutely includes a dynamic library setup. So from what I can see unless you perform the hack I’ve mentioned above, no way could you write a functional third party plugin for the world on github that includes a .dll and expect anyone to be able to just drop it in to their project.

And that really begs the question, why did the Unreal folks include in their ThirdPartyPlugin example a situation (dynamic library) that essentially will always fail? I’d sure like to speak with that person for five minutes. (And when I say “always fail” I really mean there appears no easy way to generate the correct folder structure for an “aftermarket” plugin with dynamic library via normal scripting.) Note, I do see this tool, and it has a generate Plugin tab… I have not yet downloaded that and tried it. (Oops. Upon reflection… if you wanted to publish your Plugin via the Unreal marketplace, I suspect that would then get built correctly with dynamic libraries in the Binaries folder as displayed in the example.) I wish I understood where that folder generation is taking place…

I will say to the unreal team, just a couple of well placed comments in the code sheet would go a long way. e.g.


// This example includes dynamic libraries as intended for a plugin that is distributed thru the Unreal Marketplace
// For manually installed third party plugins, best to stick with static libraries.

And I’m now realizing why lots of entries in the internet are saying “Static libraries only” for writing your own Third Party plugin…

Frankly I couldn’t get over the “Where does that Binary / ThirdParty folder get generated?” question.

One thing you mentioned… the config.ini files… I don’t see anything on the *.ini that remotely deals with plugin stuff… Nothing that would affect this ThirdPartyLibrary example from Unreal that I can see. Which folder/file might have an effect here? thanks.

7

/Config/FilterPlugin.ini is where you can define dependencies to make sure that Marketplace’s build system will include the directories and files required to ship in the package;

Example:



[FilterPlugin]
/ThirdParty/Roslyn/*.dll

/Binaries/MagicNode.dll
/Binaries/UnrealEngine.dll
/Binaries/mono-2.0-sgen.dll

/Binaries/Mono/*.*
/Binaries/Editor/*.*

/Content/Examples/*.uasset
/Content/Database/*.uasset
/Content/Library/*.uasset
/Content/Template/*.txt
/Content/Icons/*.png
/Content/UI/*.png
/Content/UI/*.ttf

The marketplace process is going to bundle all these files I have requested;
I add all them several DLLs to Build.cs and later in C++ I actually load them, like this one:



PublicDelayLoadDLLs.Add("mono-2.0-sgen.dll");

In C++ I copy few DLLs around on user project startup, but this is for a complex setup, usually you shouldn’t do this:



void FMagicNode::StartupModule()
{

    IPlatformFile & PlatformManager = FPlatformFileManager::Get().GetPlatformFile();

#ifdef MCS_NOT_SUPPORTED

   LOG::CS(EScriptSeverity::Fatal,TEXT("C# Magic Node plugin is not currently supported on target platform!"));
#else
   MonoCore_INIT();
#endif

#ifdef MCS_MARKETPLACE_BUILD
   static FString DIRxCFG = FPaths::Combine(*FPaths::EnginePluginsDir(),TEXT("Marketplace"),CS_PLUGIN_NAME,TEXT("Binaries"));
#else
   static FString DIRxCFG = FPaths::Combine(*FPaths::ProjectPluginsDir(),CS_PLUGIN_NAME,TEXT("Binaries"));
#endif

static FString DIRxDOM = FPaths::Combine(*DIRxCFG,TEXT("MagicNode.dll"));

static FString BINxDOM = FPaths::Combine(*CS_BINARY_DIR,TEXT("MagicNode.dll"));


#if WITH_EDITOR
if (!PlatformManager.DirectoryExists(*CS_SCRIPT_DIR))
{
   PlatformManager.CreateDirectory(*CS_SCRIPT_DIR);
}

if (!PlatformManager.DirectoryExists(*CS_BINARY_DIR))
{
   PlatformManager.CreateDirectory(*CS_BINARY_DIR);
}


check(PlatformManager.CopyFile(*BINxDOM,*DIRxDOM));

#endif
}

Then I let the engine solve paths and load DLLs from correct path, depending on if this is a “Marketplace Build” or a standalone plugin install:



static FString DIRxCLR = FPaths::Combine(*DIRxCFG,FString::Printf(TEXT("mono-2.0-sgen%s"),*EXT));

CLR = FPlatformProcess::GetDllHandle(*DIRxCLR);

Btw, I do not use “Plugin Generator” code… I consider them having a few bad practices, so I just go and write stuff by hand.

8

You rock. Of course it will take me days to remotely understand all the details here, but I appreciate the rapid response… Many thx…

9

Not saying Bruno’s stuff wasn’t totally off base, but it didn’t really help me understand how to really add a third party tooling to Unreal. I would like to offer up these invaluable resources that I’ve run into along the way…

  1. Cross-platform library integration in Unreal Engine 4, by Adam Rehn, 12 April 2018 This is a really nice discussion on a proposed workflow for simplifying the integration of third-party libraries into projects built with Epic Games’ Unreal Engine 4. The article has now been superseded, but this writeup helps provide some insight as to what’s going on behind the scenes.

  2. Introduction to conan-ue4cli, by Adam Rehn. This is the replacement document for that posted above. It’s pretty involved.

  3. Integration Unreal Engine 4 with CMake project and boost library from QuickTurnStudio This is an introduction as to a particular approach to address CMake issues for third party library plugins.

  4. Build and static link ZeroMQ on Windows by Joshua Burkholder This example has absolutely nothing to do with Unreal, but instead gives insight as to how to implement a static third party tool in a “standard” Visual Studio C++ implementation. There are some odd tricks to make that work, and those odd tricks help figure out whatever library you might be thinking about implementing as a UE4 third party plugin.

  5. I ran into this feedback from ScatteredRay on Discord: ScatteredRay — 4/12/2021 at 12:20 PM [NNG channel] Yeah, I’m using it in a UE4 plugin… Essentially I have a powershell script that builds the cmake, which is called from PreBuildSteps in my .uplugin. then in a *ThirdParty.Build.cs I needed to add a PublicIncludePaths and PublicAdditionalLibraries for nng.lib Essentially the pattern for implementation is the same as that used in the “sdl_build” branch of this repo: GitHub - ScatteredRay/UE4SDLController at sdl_build
    Check the XXX.uplugin, XXX…Build.cs and XXX.ps1 files contained within. That was pretty helpful.

Meanwhile, I’m still pluggin along. Not quite there yet with my tooling, but I’m hopeful. Thanks for listening.

4 Likes
10

If you’re looking for how to link a static lib from plugins, you just have to add its headers to PublicIncludePaths and the *.lib path to PublicAdditionalLibraries list in plugin’s Build.cs.

It’s much easier than dealing with dynamic libs.

11

I ran into the post looking for the ability to add cmake libraries. Those articles linked above also never really matched what I wanted as I handle most of the development in the cmake lib and the only UE4 parts were really just wrappers. In the end I ended up writing my own solution GitHub - caseymcc/UE4CMake: Provides a simple way to add a cmake lib to any Unreal Engine 4 (UE4) project., only tested with windows builds thus far but seems to work fine. It generates a cmake file that includes the original cmake library along with some helpers that pull out the needed include directories and libraries build by the lib. It also tags the project CMakeLists.txt (as well as the library source) as dependencies so if any of them change the cmake build will be re-issued when the uproject is built.

Adding it here in case it helps anyone else.

12

Struggling to comphrehend this concept, but here’s another article I came across to help better understand

2 Likes