Unit Tests in Unreal – pt 2 – Integrating Google Test with Unreal Engine

Introduction

The idea of this part is to discuss how to integrate Google Tests with Unreal Engine. For this posts, I’ve used Unreal Engine 4.21.0. You can register at Epic website and they will give you access to their GitHub repo (https://github.com/EpicGames/UnrealEngine). You will need the Engine Source code, because of the dependencies on the build tool and the base classes of the engine.

It is important to remember that Unreal Engine is not open source, so, there are licensing concerns if you build your game using Unreal, which is not my intention to discuss in this post.

If you are puzzled about the motivations for using Google Test and not Unreal Test Framework, please read the part 1.

Creating your Example Game

When you get a clear installation of Unreal Engine, you need to compile it locally. The process is to run:

  • Setup.bat: Will install all the pre-reqs you need.
  • GenerateProjectFiles.bat: Will generate the solution files and project files
  • Compile the solution in Visual Studio in Development Win64. I’ve used Visual Studio 2017. It takes a while to finish (sometimes hours).
  • If you are lucky, this will produce an UE4Editor.exe in your Engine\Binaries\Win64 dir.
  • If you are not lucky, it is very likely that you are missing any pre-requisite, but the error messages are usually not very friendly.
  • Run UE4Editor.exe
  • In the start screen, go to new project, you will get something like this

unreal-gtest1

  • Select your project directory (I used the same directory of the Engine), type C++, Basic Code, Desktop/Console, No Starter Content. The name, I’ve used ExampleGame instead of MyProject, but this should be your game.
  • You can now reopen your solution, and you will realise there’s a project for your game there. Build again in Development Editor/Win64 and you will have an Unreal Editor for your game.

A quick overview of Unreal Build Flavours

This is far away from a complete view of the build flavours, but can give you an idea.

  • Debug: Will do a monolithic build (everything bundled in a single .exe) with full debug information.
  • Development: Will do a modular build (every module is a dll) with full debug information.
  • Shipping: Will do a monolithic build without debug information.

The hot reload feature although great sometimes seems to not work properly and it is a bit hard to debug. In this cases, it is interesting to do a Debug build which will take the hot reload variable out of the game.

Each one of the main versions, like Debug will have 4 other targets. An unnamed one (just Debug), Client, Server and Editor. Client usually doesn’t work, since the one that builds the game is the unnamed one. Server turns on the WITH_SERVER define, which makes the output as the multiplayer server and Editor turns on the WITH_EDITOR define which enables the editor extensions which allows you to compile and change your on-the-fly using the hot reload.

Adding Google Tests as a 3rd party module

In order to compile and run Google Tests, you will need to add them as a module. Download from Google Test (https://github.com/abseil/googletest/releases) a version of the source code, inside ExampleGame\Source\ThirdParty\GoogleTest\googletest-release-1.8.1.

Then you should create the file ExampleGame\Source\ThirdParty\GoogleTest\GoogleTest.Build.cs with the content:

using UnrealBuildTool;
using System.IO;

namespace UnrealBuildTool.Rules {

	public class GoogleTest : ModuleRules
	{
		public GoogleTest(ReadOnlyTargetRules Target) : base(Target)		
		{
			Type = ModuleType.External;
			
			string googleTestBasePath = Path.Combine(ModuleDirectory, "googletest-release-1.8.1");

			PublicSystemIncludePaths.Add(Path.Combine(googleTestBasePath, "googlemock"));		
			PublicSystemIncludePaths.Add(Path.Combine(googleTestBasePath, "googlemock", "include"));		
			PublicSystemIncludePaths.Add(Path.Combine(googleTestBasePath, "googletest"));		
			PublicSystemIncludePaths.Add(Path.Combine(googleTestBasePath, "googletest", "include"));						
			
			PublicDefinitions.Add("GTEST_OS_WINDOWS=1");			
			PublicDefinitions.Add("GTEST_OS_WINDOWS_MOBILE=0");
			PublicDefinitions.Add("GTEST_OS_LINUX_ANDROID=0");
			PublicDefinitions.Add("GTEST_OS_LINUX=0");
			PublicDefinitions.Add("GTEST_OS_MAC=0");
			PublicDefinitions.Add("GTEST_OS_HPUX=0");
			PublicDefinitions.Add("GTEST_OS_QNX=0");
			PublicDefinitions.Add("GTEST_OS_FREEBSD=0");
			PublicDefinitions.Add("GTEST_OS_NACL=0");
			PublicDefinitions.Add("GTEST_OS_NETBSD=0");
			PublicDefinitions.Add("GTEST_OS_FUCHSIA=0");
			PublicDefinitions.Add("GTEST_OS_LINUX_ANDROID=0");
			PublicDefinitions.Add("GTEST_OS_SYMBIAN=0");
			PublicDefinitions.Add("GTEST_OS_WINDOWS_MINGW=0");
			PublicDefinitions.Add("GTEST_OS_WINDOWS_PHONE=0");
			PublicDefinitions.Add("GTEST_OS_WINDOWS_RT=0");
			PublicDefinitions.Add("GTEST_OS_CYGWIN=0");
			PublicDefinitions.Add("GTEST_OS_SOLARIS=0");
			PublicDefinitions.Add("GTEST_OS_WINDOWS_TV_TITLE=0");
			PublicDefinitions.Add("GTEST_OS_IOS=0");
			PublicDefinitions.Add("GTEST_OS_AIX=0");		
			PublicDefinitions.Add("GTEST_OS_ZOS=0");		
			
			PublicDefinitions.Add("GTEST_DONT_DEFINE_FAIL=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_SUCCEED=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_ASSERT_EQ=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_ASSERT_NE=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_ASSERT_LE=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_ASSERT_LT=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_ASSERT_GE=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_ASSERT_GT=0");
			PublicDefinitions.Add("GTEST_DONT_DEFINE_TEST=0");		
			
			PublicDefinitions.Add("GTEST_HAS_DOWNCAST_=0");
			PublicDefinitions.Add("GTEST_HAS_MUTEX_AND_THREAD_LOCAL_=0");
			PublicDefinitions.Add("GTEST_HAS_NOTIFICATION_=0");
			PublicDefinitions.Add("GTEST_HAS_ABSL=0");
			PublicDefinitions.Add("GTEST_HAS_GETTIMEOFDAY_=0");		
			
			PublicDefinitions.Add("__GXX_EXPERIMENTAL_CXX0X__=0");
			PublicDefinitions.Add("GTEST_USES_PCRE=0");		
			PublicDefinitions.Add("GTEST_USES_POSIX_RE=0");		
			PublicDefinitions.Add("GTEST_ENV_HAS_TR1_TUPLE_=0");				
			PublicDefinitions.Add("GTEST_LINKED_AS_SHARED_LIBRARY=0");
			PublicDefinitions.Add("GTEST_CREATE_SHARED_LIBRARY=0");				
			PublicDefinitions.Add("GTEST_CAN_STREAM_RESULTS_=0");
			PublicDefinitions.Add("GTEST_FOR_GOOGLE_=0");
			PublicDefinitions.Add("GTEST_GCC_VER_=0");			
			
			PublicDefinitions.Add("WIN32_LEAN_AND_MEAN=1");
		}
	}
}

Google Test can be compiled adding their source files to your project (which will be embedded in your test executable without intermediate .lib) or using cmake (which will produce a .lib that will then be linked in your executable). If you do by cmake, you should just follow gtest build instructions, generate the project files, link and tweak the options described above to link to an existing .lib instead of building from source. I’ve choosen the option 1, including their source files in my project.

The code above can be scary if you don’t know what it means, but basically because Google Test can target a lot of platforms, it needs a lot of defines to work. Because by default, Unreal is setting warnings as errors, it requires all the defines to be there. All of those definitions are from Google Test, just to make it build and work on Windows, except for WIN32_LEAN_AND_MEAN, which is needed for Windows includes.

So, Module.External means that this module is external source code. It is not a program, a game, or an editor. The PublicSystemIncludePaths are just to make the Google Test and Google Mock includes (which is included in Google Tests) to be visible by other modules. We will need that for our test projects. And everything else are just defines to satisfy the compilation of Google Test.

Create a test console application

To create a game project first, you will need to create a GoogleTestApp.uproject file in the ExampleGame directory. This is just to make your project “discoverable” by Unreal. It calculates the Intermediate, Binaries and Source folders based on this. The content can be something like:


{
	"FileVersion": 3,
	"EngineAssociation": "",
	"Category": "",
	"Description": "",
	"Modules": [
      {
        "Name": "GoogleTest",
        "Type": "Runtime",
        "LoadingPhase": "Default"
      },  
	],
	"Plugins": [
		
	],
	"TargetPlatforms": [
		"WindowsNoEditor",
		"XboxOne",
		"UWP64"
	]
}

As described in the post “Understanding Unreal Build Tool“, a target is a new compilation flavour. Because we will need a whole new process to host the tests, which will be a console application, we need to create a target. In order to do this, we will place a file GoogleTestApp.Target.cs in Game\Source directory. This file will have the content:


using UnrealBuildTool;
using System.Collections.Generic;
using System.IO;

public class GoogleTestAppTarget : TargetRules
{
	public GoogleTestAppTarget(TargetInfo Target) : base(Target)
	{
        	Type = TargetType.Program;
		LinkType = TargetLinkType.Modular;
		LaunchModuleName = "GoogleTestApp";		

		bIsBuildingConsoleApplication = true;		
	}
}

The interesting thing here is the bIsBuildingConsoleApplication. This will make UnrealBuildTool add the proper defines to expect a console application instead of a windows application.

The next step is to create the GoogleTestApp, which is the “launch” module specified above. This is the source code that will be added to this console application. So, we will create a file named GoogleTestApp.Build.cs in Source\Tests\GoogleTestApp. I’ve called “Tests” this main folder that groups all test projects, could be anything else. This file will have the content:


namespace UnrealBuildTool.Rules {
	public class GoogleTestApp : ModuleRules
	{
		public GoogleTestApp(ReadOnlyTargetRules Target) : base(Target)		
		{			            
			PrivateDependencyModuleNames.AddRange(
				new string[]
				{					
					"GoogleTest"
				}
			);
			PrivatePCHHeaderFile = "Private/GoogleTestApp.h";
		}      
	}
}

This defines that this module, GoogleTestApp depends on the module GoogleTest, which will make the headers of GoogleTest visible to GoogleTestApp. PrivatePCHHeaderFile is just the header file that will include the precompiled headers.

You should also create Source\Tests\GoogleTestApp\Private, the following files:

GoogleTestApp.h (the PCH one), with the content:

#pragma once

And GoogleTestApp.cpp, with the content:

#include "GoogleTestApp.h"

#include "gtest/gtest.h"
#include "src/gtest-all.cc" //Do not do this in Unreal 4.20. It will cause some errors because internally google test adds windows.h which conflicts to other headers that CoreMinimal.h depends.
#include "src/gmock-all.cc"
#include "src/gtest_main.cc"

class GivenASampleTest : public ::testing::Test {

};

TEST_F(GivenASampleTest, WhenRunningSampleTestShouldPass) {
	ASSERT_TRUE(true);
}

This file, will include the gtest headers, the gtest source files (gtest-all.cc include all GoogleTest source code) and gtest_main.cc. gtest_main.cc contains a definition of a function “main” which will be the entry point of the executable that runs all Google Tests. You also can remove that include and create your own main function if you want to.

The next step to validate all of this is to run GenerateProjectFiles. You should see a project GoogleTestApp created in Visual Studio. You may have errors that looks like C# compilation errors. They can be and probably this means that your .Build.cs or .Target.cs has issues.

If you use Visual Studio 2017 with GoogleTest runner, you will see that the tests are discoverable and runnable by the IDE:

unreal-gtest2

Also, you can run your tests, through command line, in ExampleGame\Binaries\Win64\ExampleGoogleTests.exe. Google test by default give you a lot of options like filters, etc. This is how you are going to run the tests in your continuous integration system (TeamCity, Jenkins, Visual Studio Team Services/Azure DevOps, etc).

Conclusion

After concluding these steps, you should have an example game in Unreal and a empty unit test project. Seems like not a lot of progress, but we will start to understand the testing challenges in the following posts.

18 thoughts on “Unit Tests in Unreal – pt 2 – Integrating Google Test with Unreal Engine

  1. Hi Eric, amazing blog!

    I’m trying to generate the project files for visual studio but it is failing with the following error:

    Running C:/Users/Giskard/Documents/GitHub/UnrealEngine/Engine/Binaries/DotNET/UnrealBuildTool.exe -projectfiles -project=”C:/Users/Giskard/Documents/Unreal Projects/bomberman/GoogleTestApp.uproject” -game -engine -progress -log=”C:\Users\Giskard\Documents\Unreal Projects\bomberman/Saved/Logs/UnrealVersionSelector-2019.05.27-15.23.47.log”
    Discovering modules, targets and source code for project…
    ERROR: Not expecting project C:\Users\Giskard\Documents\Unreal Projects\bomberman\Intermediate\ProjectFiles\GoogleTestApp.vcxproj to already have a Game/Program target (BombermanTarget) associated with it while trying to add: GoogleTestAppTarget

    I’m placing the GoogleTestApp.Target.cs in the bomberman/Source folder. If i take it out of that folder the project files are generated just fine, but then the project wont be able to find the googletest headers.

    Do you have any idea of what could be wrong? (Using 4.22)

    1. Hey Juan,

      Based on the error message, it seems like you have more than one target type “Game”. The Unit Test project is a “Program”. That may be the problem.

      It is really painful to understand how Unreal Build Tool works and I suggest you to read what I wrote. Took me hours to figure out. Where the files are matters, the content of the files matters and there’s a lot of other hidden settings that influences how the projects are compiled.

      Worst case scenario, debugging UnrealBuildTool will give you all the answers. The UnrealBuildTool code is in C# and is provided with the engine.

      1. Thank you Eric,

        It seems that having bot a Game and a Program is the problem. I’ll have to check the UBT in detail.

  2. Hi Eric,

    Thank a lot for this serie of articles, really inspiring and I definitively trying to use this principles now in my projects.

    I try to integrate the GoogleTestApp you’re talking about here, but I struggle with errors when trying to GenerateProjectFiles.

    I am not sure I’ve done things correctly and before digging to far in the UnrealBuildTool, I hope you can have a quick clue on how to fix it.

    I create a light repo here to show you what I have done https://github.com/NansPellicari/UE4WithGoogle and a pullrequest with some comments to clearly see how I integrate the modifications you talking about https://github.com/NansPellicari/UE4WithGoogle/pull/1.

    Thanks again !

    1. Hi Nans,

      I’ll make some comments in your project. I’m sorry it is being over a year I last worked with this and is definitely not fresh in my mind. Since you put some effort to create an example, that encouraged me to give some help as well. I’ll comment some things on your PR to try to help, but unfortunately I don’t have loads of time to invest on this.

      I wanted to publish a full working example, but because of the licensing of the engine I wasn’t sure if I’m allowed or not. Also it is important to notice that Unreal Engine changes a lot between versions and this applies for UnrealBuildTool as well.

      I think the other post I have: Understanding Unreal Build Tool can be of a help here.

      I’ll leave some comments in your project/PR.

      Regards,

      Eric

  3. Hi Eric,
    Just to let you know, we’ve asked Epic about license terms here https://answers.unrealengine.com/questions/950316/can-i-use-shootergame-content-in-video-course-for.html.
    – It seems that there is no legals restrictions of creating a public repository with our own code.
    – Any kind of revenue generated with our project are subject to the 5% royaltiy (eg tuition fees).
    – According to the section 1(A)e in the EULA license, we can “distribute or sublicense Examples (including as modified by us under the License) in Source Code or object code format to any third party.”, eg StartPack content, but NOT ShooterGame content.
    – ShooterGame content for example can be distributed only to other UE licensees. Forking the UnrealEngine repo allows to manage user restrictions.
    – By the way, all of the 1(A) is about distribution and sublicensing and it is pretty clear.

    Hope it’s help 🙂

    1. Hi Nans,

      I was expecting that. My main issue is distributing the Engine code. I don’t believe we are allowed to do that. One workaround would be creating a submodule pointing at Epic’s main Engine repo. Than we can point out the example to a version of the engine where everything compile and works at the same time that we are sure that whoever is checking out that submodule has been through Epic’s process to be authorised in their GitHub repo.

      If you create that link, and making sure you have a directory structure that your example more or less compile, that will be a great motivator for me to give a stab and try to make that work. 😉

      Cheers

      Eric

      1. Hi Eric,

        Sorry for my late, I was stuck with my project last week.
        Ok I make it work today https://github.com/NansPellicari/UE4WithGoogle!
        The thing is: I still can’t generate `GoogleTestApp.uproject` Visual Studio files (more details in the README.md file). It’s not such a big deal to me ’cause I don’t use Visual Studio and I prefer to run tests in terminal, but it shoulds be a problems to VS aficionados.

        I’ll look later (maybe next week) for this part, except if you have a clue?
        In the meantime I’m looking to get a “good looking” reports page and hoping finding a way to get code coverages.
        But for now I can carry whats already work, i’m so glad, the speed of the run it’s soooo fast, it’s really awesome :)!

        About your last reply:
        – I tried to create an UnrealEngine submodule directly at the root of the project
        – then (after a loooong download & build process) I tried to build the game project but UE doesn’t like to have itself in a project it builds (it removes some source and third party files, it’s a complete mess…)
        – Then before I changed the structure of the repo to have the UE submodule and the project side by side, I realized: “But… the repo doesn’t exposed Engine code!?” 😀
        Or maybe I miss something ?

        A BIG thanks! This will increase so much my development process and quality of the project, I’m really happy with that!

      2. Hi Nans,

        Regarding engine/game code, if you move all your source inside Game/Source and create the submodule on an Engine folder, it should work fine. It does take ages to download the source code is kind of expected. I’m suggesting this approach so you can point out to a commit/version of the engine that you know it works as usually it has some breaking changes.

        If you do that or give me which version/commit of the engine you are using, I can give a stab, but I can’t promise you when since I’m a bit swamped with a lot of random things.

        Glad to hear that it is moving forward and thanks for putting this together.

        If you want to get some coverage, check out OpenCppCoverage.

        Regards,

        Eric

      3. Oh yeah sure I can do that if it’s easier to manage. If i’ve well understand: create a branch on my UnrealEngine’s fork and embed the project as a submodule in a dedicated Engine’s subfolder?
        For now to build the project I used the commit feeb3c7469e on the “release” branch, this ci is flagged by the “4.24.3-release” tag.

        Sure don’t worry, take your time! I’m already fine with the actual solution, i can make really fast proof of concepts which was my early goal. I’m very new on cpp programming, so i do a lot of stupid mistakes and having this GGtests env helps me a lot already!

        Thanks for the OpenCppCoverage tips, I will test it soon!

        Regards,

  4. Hi Eric,

    I created this repo to provide a project’s template to simplify the use of GG and UE4 tests and having nice reports pages: https://github.com/NansPellicari/UE4-TPL-CppWithTestEnv.
    It also provides the capacity of code coverage using OpenCppCoverage as you suggested !
    Before showing it to the UE4 community, do you mind if I mentionned you and these series of articles In the “1. Why ?” section?

    Regards,

    1. Hi Nans,

      Of course not. I appreciate if you reference the original articles and keep me in the loop. I believe this kind of collaboration is a win win. I really appreciate you getting involved and adding more capabilities and insights in the mix.

      BTW. “Where” is Unreal Engine community? I’d like to have a look on what is going on there.

      Regards

      Eric

  5. First of all, thank you so much for this guide, it was really helpful!
    And thanks also to npellicari for the github repo, great work!
    I have one question though: I noticed that latest version of unreal has a folder called GTest in its ThirdParty. Specifically this path:
    ..\Epic Games\UE_4.24\Engine\Source\ThirdParty\GoogleTest
    Did you have a look at this? Do you think Epic is trying to integrate GoogleTest in Unreal? I tried to use it but apparently it doesn’t build if I include gtest paths. It sees all the sdk folders but it seems that all the paths for the various headers inside are all broken… I wonder if this is still work in progress. What do you think?

    1. Hi marcomal,

      Yes, they keep a version of GTest there that should be possible to integrate. Usually these broken headers means wrong configuration on .Target.cs or .Build.cs. I won’t give an opinion about what is wrong without trying which I to be honest will not do anytime soon. But this https://ericlemes.com/2018/11/23/understanding-unreal-build-tool/ definitely help you to figure out what is wrong. What I can’t say if there is other module making use of this GTest with this configuration. You can find that out checking the module configurations (.Build.cs) of the whole Engine. It is a bit of a painful process, but once you understand how Unreal deals with your compiler it is possible (after knocking your head on the wall in desperation a few times).

      1. The issue there is exactly what I mentioned. It seems that the original GTest added there has wrong .Build.cs information. The include paths (based on https://ericlemes.com/2018/12/13/unit-tests-in-unreal-pt-2/) is done by these lines:

        PublicSystemIncludePaths.Add(Path.Combine(googleTestBasePath, “googlemock”));
        PublicSystemIncludePaths.Add(Path.Combine(googleTestBasePath, “googlemock”, “include”));
        PublicSystemIncludePaths.Add(Path.Combine(googleTestBasePath, “googletest”));
        PublicSystemIncludePaths.Add(Path.Combine(googleTestBasePath, “googletest”, “include”));

        So, probably in the original .Build.cs the paths doesn’t match what is the disk. This has anything to do with Google Test but how unreal engine handles include path and C++ compilation as whole. You have 2 ways forward. Trying to fix their .Build.cs, or follow from scratch what is on the tutorial part 2. Both ways will give you the same result (probably with different versions of Google Test though).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s