How to integrate third party unit testing with modules

Development Programming & Scripting C++
1

I took me a good chunk of time to figure out how to do this, and finding information on the various parts was infuriating at times, so I want to share what I did in case it may help someone. If there are parts that may be improved, I would of course appreciate the aid. I am using the source 4.7.6 engine.

It should be said that I am insulating myself pretty heavily from the Unreal code, using as little UObject, Unreal macros, and so on as possible. Sorry if this ruins applicability for people.

I wanted unit testing, but disliked Unreal’s version. I wanted something lightweight and independent of Unreal’s boilerplate (because of my insulation), and settled on Catch. Integrating it is as simple as adding one hpp file and giving access to the main(). I wanted the test code distinctly separated from the game code. With help from the Answers site, I went with making a separate executable module.

I want to test the internal code and not any UObjects. I don’t want the testing module to need the engine or editor to run, and I figure I can mock any of the wrappers I make fairly easily. So I settled on three modules: the primary module MyProject handles any interactions with the engine and editor, MyCoreModule does all of the important internal game stuff, and MyTestingModule tests only MyCoreModule. MyProject is the primary game module, and actually has the name of the project itself.

Here is a link to the source. Hopefully this will make it easier to follow along.

[HR][/HR]
Directory structure:

Project
Source
[INDENT]MyProject.Target.cs
MyProjectEditor.Target.cs
MyProject
[INDENT]MyProject.Build.cs
MyProjectGameMode.h
MyProjectGameMode.cpp
MyProject.h
MyProject.cpp
WrapperClasses
[INDENT]EntityMediator.h
EntityMediator.cpp
EntityWrapper.h
EntityWrapper.cpp

[/INDENT]
MyCoreModule
MyCoreModule.Build.cs
MyCoreModule.h
MyCoreModule.cpp
CoreClasses
[INDENT]Entity.h
Entity.cpp
IEntityMediator.h

[/INDENT]
[/INDENT]
[/INDENT]

UnrealSource
Engine
[INDENT]Source
[INDENT]Programs
[INDENT]MyTestingModule
[INDENT]MyTestingModule.Build.cs
MyTestingModule.Target.cs
catch.hpp
MyTestingModule.h
MyTestingModule.cpp
TestClasses
[INDENT]TestEntity.h
TestEntity.cpp

[/INDENT]
[/INDENT]
[/INDENT]
[/INDENT]
[/INDENT]

The testing module may also be placed in a subdirectory Programs of MyProject’s source.

Let me give a schematic of steps to get everything working from a new project. First, let’s get MyCoreModule up and connected to MyProject.

[HR][/HR]
MyCoreModule:

The base MyCoreModule files (MyCoreModule.Build.cs, MyCoreModule.h, MyCoreModule.cpp) have the same structure as the default MyProject files. Since I’m only using structs from Unreal for now, only Core is needed as a public dependency in Build.cs. However, it still needs to find the file ModuleManager.h in Runtime/Core/Projects/Modules/, so add that as a public include path. MyCoreModule does not need Engine, so MyCoreModule.h includes only Core.h. MyCoreModule.cpp includes MyCoreModule.h and the aforementioned ModuleManager.h, and this is not the primary module, so IMPLEMENT_GAME_MODULE instead of IMPLEMENT_PRIMARY_GAME_MODULE.

Now in the primary module, MyCoreModule should be added to the modules in MyProject.uproject (I use the PreDefault loading phase), to OutExtraModuleNames in MyProject.Target.cs and MyProjectEditor.Target.cs, and to PrivateDependencyModuleNames in MyProject.Build.cs. To be honest, I am not sure how public/private dependency affects things; it may be best to add it to PublicDependencyModuleNames instead.

At this point, regenerate the project files and the modules should link up appropriately. I have added several files in subfolders to demonstrate communication between the wrapper and core modules. Note that any class or function included across the modules should have the MYPROJECT_API or MYCOREMODULE_API macros. Since MyCoreModule should not be referencing MyProject at all, the latter macro on appropriate classes should do.

[HR][/HR]
MyTestingModule:

The testing module is a bit different and will produce a standalone executable. It is placed in a Programs subdirectory of either the engine’s or project’s Source and will make its own VS project when the project files are generated.

Unlike MyCoreModule, it needs its own Target.cs with nontrivial content. The TargetType is Program instead of Game. The UEBuildBinaryConfiguration needs an Executable InType and includes both MyTestingModule and MyCoreModule in InModuleNames. I am not sure how many of the options I chose are necessary, but I compile monolithic, lean and mean, without editor or editor-only data, and don’t compile against Engine or CoreUObject. See the BlankProgram program for comparison.

MyTestingModule.Build.cs needs the project source directory in its include paths, as well as the engine’s Runtime/Launch/Public and Runtime/Launch/Private directories. The private dependencies are Core, Projects, and MyCoreModule.

Place catch.hpp in the base directory (or anywhere, I suppose, as long as you can find it). For catch to work, it just needs to run in the main method. So in MyTestingModule.h, include Core.h and catch.hpp. However, catch uses Windows specific stuff, so the include needs to be wrapped between #include “AllowWindowsPlatformTypes.h” and #include “HideWindowsPlatformTypes.h”.

MyTestingModule.cpp is where main() runs. Define catch’s CATCH_CONFIG_RUNNER at the top. Include MyTestingModule.h of course, and also RequiredProgramMainCPPInclude.h since this is a program. Use the IMPLEMENT_APPLICATION macro. Unreal’s main() macro has already converted from ANSICHAR to TCHAR, and catch needs the original main(int32 argC, ANSICHAR* Utf8ArgV]). So I found the macro definition and extracted it so that I could do the conversion and make Unreal’s tchar_main method but still see the original ANSICHAR for catch. One final point is that there is a strange std::_Tree_node<…> error as is. StackExchange has given me the impression this is a VS bug, and disabling the warning with #pragma warning (disable:4610) has not caused problems so far.

At this point, regenerating the project files should put MyTestingModule as a new project under programs. Make sure that it is building correctly under the Configuration Manager (Development_Program, build checked). I added some files to show how to test, including a mock of the wrapper.

[HR][/HR]
Problems and uncertainties:

Unfortunately, if the program links to MyCoreModule, it builds the executable into MyProject’s Binaries. This would be fine, and in fact preferable, except that the executable seems to only work if in the engine’s Binaries directory. If anyone knows how to target the engine’s Binaries or, better, how to set up the executable so that it runs from the project’s Binaries, I would appreciate knowing.

I do not know how much of the required Public/Private directory structure in Unreal is legacy. I did not use any of this structure, and I don’t know if that is fine, or if there is some necessary interaction between this structure and the Public/Private Include/Dependency code in Build.cs, or something else entirely.

It would be nice to know what options in MyTestingModule.Target.cs are necessary/preferable/meaningless.

[HR][/HR]
I hope that this can help someone getting started with modules, unit testing, integrating third party code, etc. Problems nonwithstanding, I am finally starting to feel comfortable in my setup. Best wishes!

2

Due to changes in the UnrealBuildTool, the method described above no longer works as described. Two issues:

There is probably a way to leave the testing source in UnrealSource, but I couldn’t figure it out. Put the MyTestingModule folder in the Programs subfolder of your game’s Source.

The Build Tool has been changed to only look for Target.cs files in a source directory recursively until it finds a set. Since the game’s Target.cs files are in Project/Source and the testing Target.cs files will be in Project/Source/Programs/TestingModuleName, the Build tool will look in GameProject/Source and stop as soon as it finds the game’s target files. After digging to find all of this, the solution is fairly simple.

The method FindTargetFiles in UnrealSource/Engine/Source/Programs/UnrealBuildTool/System/UProjectInfo.cs searches directories recursively for Target.cs files. Changing the code to search until it finds all files is very simple. Here is the new method.



public static bool FindTargetFiles(string InCurrentTopDirectory, ref bool bOutFoundTargetFiles)
{
    string CurrentTopDirectory = InCurrentTopDirectory;
    List<string> SubFolderList = new List<string>();

    // Check the root directory
    bOutFoundTargetFiles |= FindTargetFilesInFolder(CurrentTopDirectory);

    // Recurse
    foreach(var TargetFolder in Directory.EnumerateDirectories(CurrentTopDirectory, "*", SearchOption.TopDirectoryOnly))
    {
        FindTargetFiles(TargetFolder, ref bOutFoundTargetFiles);
    }

    return bOutFoundTargetFiles;
}

It is possible that removing this might mess something up, and of course searching all subdirectories should take a little longer when looking, but I have had no problems with continued development so far.

3

This is exactly what I’ve been looking for – I’d like to run quite a few unit tests in isolation. Keep up the good work!

I’ll let you know how we get on with adding it to our project…

Cheers,

4

Good luck! I certainly struggled enough to get a working setup; if I can help, please let me know. There were certainly things that I tweaked for things to work just right.

5

Hello, I’m so lucky to finally have found this post!

I’ve been struggling with the same problem for about an entire week now. I actually use Catch too, so this is like heaven for me.
I may understand part of the pain you’ve gone through. After trying a lot of stuff with modules, plugins, programs… My last approach was linking the Engine in a Program inside the UE source, but after going through a hell of a lot of compilation errors and resolving each one of them by finding the correct include paths and modules… I couldn’t find a way to add my own game module as part of the Program Build.cs.
Now, I was again searching for help online, and I came across your post.

Really, thanks a lot for posting your solution, I hope the TDD workflow gets better over time within Unreal Engine :wink:

6

Hey guys, I’m facing a similar problem and found this post. I’ve went the same way basically ending up with copying BlankProgram under my project’s source tree after a lot of other failed attempts. But I’m stuck at the linking stage where the linker complains about not finding ‘TestProject\Binaries\Win64\BlankProgram-Core.lib’. For some reason UBT doesn’t do anything to build that target first. I had the same issue with my original Program Module before I felt back to BlankProgram for a clean start. Probably missing some critical step here. Any pointers?

7

Okay, figured that out. Just in case somebody else might find it useful: my problem was using Rocket. Either use the source code build of the engine or have an extra step that patches the auto-generated project files and removes -rocket from the arguments passed to UBT. Then it will correctly build the modules your code depends on. I even managed to have non-monolithic builds with Rocket that way to get faster iteration times.

8

Just an update for those of you who may be having trouble with the above steps.

I have finally got a version working locally in my Unreal Projects/MyProject folder but it took a very long time and I ran into a lot of issues. Most of which are documented in this thread, but I have slightly different work arounds.

The main pain points with creating a console application are hard coded values in the engine that expect your console application to be in the engine folder tree. Obviously that is not ideal because the unit test project should be local to the project being developed and not the engine. I also wanted to be able to painlessly migrate to new engine versions as they are released, so changing the source code of the engine was a last resort for me. One final point, I am working on two different systems with two different install locations, so hard coding the engine path was also not ideal.

Note: In the example below, I used a name of MyTests for the tests module. This module is a copy of the BlankProgram project found at <install path>/Engine/Source/Programs. You might want to use a more descriptive name (such as MyProjectTests) because temporary files will be created in the Unreal Projects/Engine/Programs folder and it might cause name conflicts if you work on multiple different games. I also didn’t link MyCore into MyTests yet, but that step should be fairly straight forward if you can manage the rest.

Custom Environment Variable

UnrealEngine = “C:\UnrealEngine\4.11.2”

Directory Structure

  • Unreal Projects

    • Engine (copy of internationalization files hard coded to be at …/…/…/Engine)

      • Binaries

        • ThirdParty

          • ICU

            • icu4c-53_1 (copied from engine source <install path>/Engine/Binaries/ThirdParty/ICU)

      • Content

        • Internationalization (copied from engine source <install path>/Engine/Content)

    • MyProject

      • Source

        • MyCore

          • MyCore.Build.cs

        • MyGame

          • MyGame.Build.cs

        • MyTests

          • MyTests.Build.cs

        • MyProject.Target.cs

        • MyProjectEditor.Target.cs

        • MyTests.Target.cs (put all .Target.cs files in same location due to the way the build tool stops recursion at odd points)

      • MyProject.uproject

      • MyTests.uproject

MyTests.uproject (not sure what parts are essential, if any, but the file needs to be found by the build tool)



{
	"FileVersion": 3,
	"EngineAssociation": "4.11",
	"Category": "",
	"Description": "",
	"Modules": 
		{
			"Name": "MyTests",
			"Type": "Runtime",
			"LoadingPhase": "Default"
		},
		{
			"Name": "MyCore",
			"Type": "Runtime",
			"LoadingPhase": "Default"
		}
	],
	"TargetPlatforms": 
		"WindowsNoEditor"
	]
}

MyTests.Target.cs



using System;
using UnrealBuildTool;
using System.Collections.Generic;

public class MyTestsTarget : TargetRules
{
	public MyTestsTarget(TargetInfo Target)
	{
		Type = TargetType.Program;
	}

	public override void SetupBinaries(
		TargetInfo Target,
		ref List<UEBuildBinaryConfiguration> OutBuildBinaryConfigurations,
		ref List<string> OutExtraModuleNames
	)
	{
		OutBuildBinaryConfigurations.Add(
			new UEBuildBinaryConfiguration(InType: UEBuildBinaryType.Executable, InModuleNames: new List<string>() { "MyTests" })
		);
	}

	public override bool ShouldCompileMonolithic(UnrealTargetPlatform InPlatform, UnrealTargetConfiguration InConfiguration)
	{
		return true;
	}

	public override void SetupGlobalEnvironment(
		TargetInfo Target,
		ref LinkEnvironmentConfiguration OutLinkEnvironmentConfiguration,
		ref CPPEnvironmentConfiguration OutCPPEnvironmentConfiguration
		)
	{
		BuildConfiguration.bUseMallocProfiler = false;

		UEBuildConfiguration.bCompileLeanAndMeanUE = true;
		UEBuildConfiguration.bBuildEditor = false;
		UEBuildConfiguration.bBuildWithEditorOnlyData = false;
		UEBuildConfiguration.bCompileAgainstEngine = false;
		UEBuildConfiguration.bCompileAgainstCoreUObject = false;

		OutLinkEnvironmentConfiguration.bIsBuildingConsoleApplication = true;
	}
}

MyTests.Build.cs (note the use of the environment variable “UnrealEngine” so that I can work with different install setups)



using System;
using UnrealBuildTool;

public class MyTests : ModuleRules
{
	public MyTests(TargetInfo Target)
	{
		string myUnrealEnginePath = Environment.GetEnvironmentVariable("UnrealEngine");
		PublicIncludePaths.Add(myUnrealEnginePath + "/Engine/Source/Runtime/Launch/Public");
		PrivateIncludePaths.Add(myUnrealEnginePath + "/Engine/Source/Runtime/Launch/Private");

		PrivateDependencyModuleNames.AddRange(new] { "Core", "Projects" });
	}
}

Edit: Should also mention I am using the source install rather than downloading the binary (as per bstone’s comment) to fix the *-Core.dll issue.

9

For anyone wanting to avoid the Environment.GetEnvironmentVariable and having to have infrastructure in place to switch that environment variable when switching projects, I found this to work instead:



using UnrealBuildTool;
using Tools.DotNETCommon;
using System;
using System.Reflection;
using System.IO;

// then down in body of constructor:
        Assembly asm = Assembly.GetAssembly(typeof(ModuleRules));
        String PublicLaunchPath = Path.GetFullPath(Path.Combine(Path.GetDirectoryName(asm.GetOriginalLocation()), "..", "..", "..", "Engine", "Source", "Runtime", "Launch", "Public"));
        String PrivateLaunchPath = Path.GetFullPath(Path.Combine(Path.GetDirectoryName(asm.GetOriginalLocation()), "..", "..", "..", "Engine", "Source", "Runtime", "Launch", "Private"));

        PublicIncludePaths.Add(PublicLaunchPath);
        PrivateIncludePaths.Add(PrivateLaunchPath); // For LaunchEngineLoop.cpp include

Thanks for this post, I needed a companion .exe for my project to ship in packaged builds and this was the best way so far I found to do it all within the project and not engine directory.