Unit Tests in Unreal – pt 2 – Integrating Google Test with Unreal Engine

Introduction

The idea of this part is to discuss how to integrate Google Tests with Unreal Engine. For this posts, I’ve used Unreal Engine 4.21.0. You can register at Epic website and they will give you access to their GitHub repo (https://github.com/EpicGames/UnrealEngine). You will need the Engine Source code, because of the dependencies on the build tool and the base classes of the engine.

It is important to remember that Unreal Engine is not open source, so, there are licensing concerns if you build your game using Unreal, which is not my intention to discuss in this post.

If you are puzzled about the motivations for using Google Test and not Unreal Test Framework, please read the part 1.

Creating your Example Game

When you get a clear installation of Unreal Engine, you need to compile it locally. The process is to run:

  • Setup.bat: Will install all the pre-reqs you need.
  • GenerateProjectFiles.bat: Will generate the solution files and project files
  • Compile the solution in Visual Studio in Development Win64. I’ve used Visual Studio 2017. It takes a while to finish (sometimes hours).
  • If you are lucky, this will produce an UE4Editor.exe in your Engine\Binaries\Win64 dir.
  • If you are not lucky, it is very likely that you are missing any pre-requisite, but the error messages are usually not very friendly.
  • Run UE4Editor.exe
  • In the start screen, go to new project, you will get something like this

unreal-gtest1

  • Select your project directory (I used the same directory of the Engine), type C++, Basic Code, Desktop/Console, No Starter Content. The name, I’ve used ExampleGame instead of MyProject, but this should be your game.
  • You can now reopen your solution, and you will realise there’s a project for your game there. Build again in Development Editor/Win64 and you will have an Unreal Editor for your game.

A quick overview of Unreal Build Flavours

This is far away from a complete view of the build flavours, but can give you an idea.

  • Debug: Will do a monolithic build (everything bundled in a single .exe) with full debug information.
  • Development: Will do a modular build (every module is a dll) with full debug information.
  • Shipping: Will do a monolithic build without debug information.

The hot reload feature although great sometimes seems to not work properly and it is a bit hard to debug. In this cases, it is interesting to do a Debug build which will take the hot reload variable out of the game.

Each one of the main versions, like Debug will have 4 other targets. An unnamed one (just Debug), Client, Server and Editor. Client usually doesn’t work, since the one that builds the game is the unnamed one. Server turns on the WITH_SERVER define, which makes the output as the multiplayer server and Editor turns on the WITH_EDITOR define which enables the editor extensions which allows you to compile and change your on-the-fly using the hot reload.

Adding Google Tests as a 3rd party module

In order to compile and run Google Tests, you will need to add them as a module. Download from Google Test (https://github.com/abseil/googletest/releases) a version of the source code, inside ExampleGame\Source\ThirdParty\GoogleTest\googletest-release-1.8.1.

Then you should create the file ExampleGame\Source\ThirdParty\GoogleTest\GoogleTest.Build.cs with the content:

using UnrealBuildTool;
using System.IO;

namespace UnrealBuildTool.Rules {

	public class GoogleTest : ModuleRules
	{
		public GoogleTest(ReadOnlyTargetRules Target) : base(Target)		
		{
			Type = ModuleType.External;
			
			string googleTestBasePath = Path.Combine(ModuleDirectory, "googletest-release-1.8.1");

			PublicSystemIncludePaths.Add(Path.Combine(googleTestBasePath, "googlemock"));		
			PublicSystemIncludePaths.Add(Path.Combine(googleTestBasePath, "googlemock", "include"));		
			PublicSystemIncludePaths.Add(Path.Combine(googleTestBasePath, "googletest"));		
			PublicSystemIncludePaths.Add(Path.Combine(googleTestBasePath, "googletest", "include"));						
			
			PublicDefinitions.Add("GTEST_OS_WINDOWS=1");			
			PublicDefinitions.Add("GTEST_OS_WINDOWS_MOBILE=0");
			PublicDefinitions.Add("GTEST_OS_LINUX_ANDROID=0");
			PublicDefinitions.Add("GTEST_OS_LINUX=0");
			PublicDefinitions.Add("GTEST_OS_MAC=0");
			PublicDefinitions.Add("GTEST_OS_HPUX=0");
			PublicDefinitions.Add("GTEST_OS_QNX=0");
			PublicDefinitions.Add("GTEST_OS_FREEBSD=0");
			PublicDefinitions.Add("GTEST_OS_NACL=0");
			PublicDefinitions.Add("GTEST_OS_NETBSD=0");
			PublicDefinitions.Add("GTEST_OS_FUCHSIA=0");
			PublicDefinitions.Add("GTEST_OS_LINUX_ANDROID=0");
			PublicDefinitions.Add("GTEST_OS_SYMBIAN=0");
			PublicDefinitions.Add("GTEST_OS_WINDOWS_MINGW=0");
			PublicDefinitions.Add("GTEST_OS_WINDOWS_PHONE=0");
			PublicDefinitions.Add("GTEST_OS_WINDOWS_RT=0");
			PublicDefinitions.Add("GTEST_OS_CYGWIN=0");
			PublicDefinitions.Add("GTEST_OS_SOLARIS=0");
			PublicDefinitions.Add("GTEST_OS_WINDOWS_TV_TITLE=0");
			PublicDefinitions.Add("GTEST_OS_IOS=0");
			PublicDefinitions.Add("GTEST_OS_AIX=0");		
			PublicDefinitions.Add("GTEST_OS_ZOS=0");		
			
			PublicDefinitions.Add("GTEST_DONT_DEFINE_FAIL=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_SUCCEED=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_ASSERT_EQ=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_ASSERT_NE=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_ASSERT_LE=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_ASSERT_LT=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_ASSERT_GE=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_ASSERT_GT=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_TEST=0");		
			
			PublicDefinitions.Add("GTEST_HAS_DOWNCAST_=0");
			PublicDefinitions.Add("GTEST_HAS_MUTEX_AND_THREAD_LOCAL_=0");
			PublicDefinitions.Add("GTEST_HAS_NOTIFICATION_=0");
			PublicDefinitions.Add("GTEST_HAS_ABSL=0");
			PublicDefinitions.Add("GTEST_HAS_GETTIMEOFDAY_=0");		
			
			PublicDefinitions.Add("__GXX_EXPERIMENTAL_CXX0X__=0");
			PublicDefinitions.Add("GTEST_USES_PCRE=0");		
			PublicDefinitions.Add("GTEST_USES_POSIX_RE=0");		
			PublicDefinitions.Add("GTEST_ENV_HAS_TR1_TUPLE_=0");				
			PublicDefinitions.Add("GTEST_LINKED_AS_SHARED_LIBRARY=0");
			PublicDefinitions.Add("GTEST_CREATE_SHARED_LIBRARY=0");				
			PublicDefinitions.Add("GTEST_CAN_STREAM_RESULTS_=0");
			PublicDefinitions.Add("GTEST_FOR_GOOGLE_=0");
			PublicDefinitions.Add("GTEST_GCC_VER_=0");			
			
			PublicDefinitions.Add("WIN32_LEAN_AND_MEAN=1");
		}
	}
}

Google Test can be compiled adding their source files to your project (which will be embedded in your test executable without intermediate .lib) or using cmake (which will produce a .lib that will then be linked in your executable). If you do by cmake, you should just follow gtest build instructions, generate the project files, link and tweak the options described above to link to an existing .lib instead of building from source. I’ve choosen the option 1, including their source files in my project.

The code above can be scary if you don’t know what it means, but basically because Google Test can target a lot of platforms, it needs a lot of defines to work. Because by default, Unreal is setting warnings as errors, it requires all the defines to be there. All of those definitions are from Google Test, just to make it build and work on Windows, except for WIN32_LEAN_AND_MEAN, which is needed for Windows includes.

So, Module.External means that this module is external source code. It is not a program, a game, or an editor. The PublicSystemIncludePaths are just to make the Google Test and Google Mock includes (which is included in Google Tests) to be visible by other modules. We will need that for our test projects. And everything else are just defines to satisfy the compilation of Google Test.

Create a test console application

To create a game project first, you will need to create a GoogleTestApp.uproject file in the ExampleGame directory. This is just to make your project “discoverable” by Unreal. It calculates the Intermediate, Binaries and Source folders based on this. The content can be something like:


{
	"FileVersion": 3,
	"EngineAssociation": "",
	"Category": "",
	"Description": "",
	"Modules": [
      {
        "Name": "GoogleTest",
        "Type": "Runtime",
        "LoadingPhase": "Default"
      },  
	],
	"Plugins": [
		
	],
	"TargetPlatforms": [
		"WindowsNoEditor",
		"XboxOne",
		"UWP64"
	]
}

As described in the post “Understanding Unreal Build Tool“, a target is a new compilation flavour. Because we will need a whole new process to host the tests, which will be a console application, we need to create a target. In order to do this, we will place a file GoogleTestApp.Target.cs in Game\Source directory. This file will have the content:


using UnrealBuildTool;
using System.Collections.Generic;
using System.IO;

public class GoogleTestAppTarget : TargetRules
{
	public GoogleTestAppTarget(TargetInfo Target) : base(Target)
	{
        	Type = TargetType.Program;
		LinkType = TargetLinkType.Modular;
		LaunchModuleName = "GoogleTestApp";		

		bIsBuildingConsoleApplication = true;		
	}
}

The interesting thing here is the bIsBuildingConsoleApplication. This will make UnrealBuildTool add the proper defines to expect a console application instead of a windows application.

The next step is to create the GoogleTestApp, which is the “launch” module specified above. This is the source code that will be added to this console application. So, we will create a file named GoogleTestApp.Build.cs in Source\Tests\GoogleTestApp. I’ve called “Tests” this main folder that groups all test projects, could be anything else. This file will have the content:


namespace UnrealBuildTool.Rules {
	public class GoogleTestApp : ModuleRules
	{
		public GoogleTestApp(ReadOnlyTargetRules Target) : base(Target)		
		{			            
			PrivateDependencyModuleNames.AddRange(
				new string[]
				{					
					"GoogleTest"
				}
			);
			PrivatePCHHeaderFile = "Private/GoogleTestApp.h";
		}      
	}
}

This defines that this module, GoogleTestApp depends on the module GoogleTest, which will make the headers of GoogleTest visible to GoogleTestApp. PrivatePCHHeaderFile is just the header file that will include the precompiled headers.

You should also create Source\Tests\GoogleTestApp\Private, the following files:

GoogleTestApp.h (the PCH one), with the content:

#pragma once

And GoogleTestApp.cpp, with the content:

#include "GoogleTestApp.h"

#include "gtest/gtest.h"
#include "src/gtest-all.cc" //Do not do this in Unreal 4.20. It will cause some errors because internally google test adds windows.h which conflicts to other headers that CoreMinimal.h depends.
#include "src/gmock-all.cc"
#include "src/gtest_main.cc"

class GivenASampleTest : public ::testing::Test {

};

TEST_F(GivenASampleTest, WhenRunningSampleTestShouldPass) {
	ASSERT_TRUE(true);
}

This file, will include the gtest headers, the gtest source files (gtest-all.cc include all GoogleTest source code) and gtest_main.cc. gtest_main.cc contains a definition of a function “main” which will be the entry point of the executable that runs all Google Tests. You also can remove that include and create your own main function if you want to.

The next step to validate all of this is to run GenerateProjectFiles. You should see a project GoogleTestApp created in Visual Studio. You may have errors that looks like C# compilation errors. They can be and probably this means that your .Build.cs or .Target.cs has issues.

If you use Visual Studio 2017 with GoogleTest runner, you will see that the tests are discoverable and runnable by the IDE:

unreal-gtest2

Also, you can run your tests, through command line, in ExampleGame\Binaries\Win64\ExampleGoogleTests.exe. Google test by default give you a lot of options like filters, etc. This is how you are going to run the tests in your continuous integration system (TeamCity, Jenkins, Visual Studio Team Services/Azure DevOps, etc).

Conclusion

After concluding these steps, you should have an example game in Unreal and a empty unit test project. Seems like not a lot of progress, but we will start to understand the testing challenges in the following posts.

3 thoughts on “Unit Tests in Unreal – pt 2 – Integrating Google Test with Unreal Engine

  1. Hi Eric, amazing blog!

    I’m trying to generate the project files for visual studio but it is failing with the following error:

    Running C:/Users/Giskard/Documents/GitHub/UnrealEngine/Engine/Binaries/DotNET/UnrealBuildTool.exe -projectfiles -project=”C:/Users/Giskard/Documents/Unreal Projects/bomberman/GoogleTestApp.uproject” -game -engine -progress -log=”C:\Users\Giskard\Documents\Unreal Projects\bomberman/Saved/Logs/UnrealVersionSelector-2019.05.27-15.23.47.log”
    Discovering modules, targets and source code for project…
    ERROR: Not expecting project C:\Users\Giskard\Documents\Unreal Projects\bomberman\Intermediate\ProjectFiles\GoogleTestApp.vcxproj to already have a Game/Program target (BombermanTarget) associated with it while trying to add: GoogleTestAppTarget

    I’m placing the GoogleTestApp.Target.cs in the bomberman/Source folder. If i take it out of that folder the project files are generated just fine, but then the project wont be able to find the googletest headers.

    Do you have any idea of what could be wrong? (Using 4.22)

    1. Hey Juan,

      Based on the error message, it seems like you have more than one target type “Game”. The Unit Test project is a “Program”. That may be the problem.

      It is really painful to understand how Unreal Build Tool works and I suggest you to read what I wrote. Took me hours to figure out. Where the files are matters, the content of the files matters and there’s a lot of other hidden settings that influences how the projects are compiled.

      Worst case scenario, debugging UnrealBuildTool will give you all the answers. The UnrealBuildTool code is in C# and is provided with the engine.

      1. Thank you Eric,

        It seems that having bot a Game and a Program is the problem. I’ll have to check the UBT in detail.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s