I was recently turned on to the idea of unit testing and test-driven development (TDD), a staple of programming and software development in other industries (to my knowledge). I know a lot of people think that it’s not possible/practical in games, but I’ve found articles that disagree (this three-part series is awesome at explaining some pros/cons with examples). As a result, I thought I’d give it a try with a UE4 project.
It’s practically impossible to unit test a UE4-based project as far as I can tell - the dependencies run too deep and are crazy to try and mock up. If I want to include/reference a class that itself includes some part of UE4, I end up with a crazy tree to try to wrestle. Add this to the fact that I can’t setup my game and my unit-test project in the same solution, since UBT/UE4 will regenerate that solution and ignore the unit test project.
Does anyone in the community unit test their projects? It seems like a very simple and logical solution to code maintainability and testing, and is something that practically everybody else likes to do, but when it’s combined with UE4 it seems like one hell of a headache.
I’ve seen some threads on the forum that play around with the idea like this one:
I think I’ve seen some others as well, but I couldn’t find them with my searches, I’m sure they’re there with the correct search terms though.
There is support for it in the engine though, you have these resources:
Like they suggest, for concrete examples in the engine, TimespanTest.cpp seems to be a good place to start.
Following this point is more of a tangental rant from my part… you have been warned >_>
I like writing tests in my day job, and I occasionally do TDD when it fits the stuff I’m doing at the moment. The problem for me is that I do a lot of experimental programming, and writing games is also very similar to that, which I guess is why most game programmers avoid it. Another problem is of course time constraints, especially within the gaming industry.
That article seems well written, and it basically condenses most of the basic principles in Kent Beck’s excellent book on the subject “Test Driven Developement: By Example”. It’s pretty old, but it’s still probably the best book I’ve read on the subject and they also mention it in the article :), but later on they also touch on the biggest problem that I also mentioned regarding experimental programming (part 3 in the series).
I don’t think I personally would recommend using it in UE4 or in game dev at all except for the things where it makes sense. Examples of that are the parts of code that are very easily specified, like your math library, your physics engine, your lighting code etc. Fortunately for us, epic has already taken responsibility for almost all of these systems, which leaves us with writing the experimental gameplay code.
So maybe if you were writing a game where you are very certain exactly how you want the game to be, and you have design docs for everything, maybe in those cases TDD would be appropriate, but then again, if you basically remove all creative freedom from the development process by doing that, where is the fun left in making the game?
I think that my preference will always be to really spend a good amount of time in making sure the architecture is simple but robust enough to keep adding features to the game without introducing bugs. And keep the code well structured enough to squash those bugs if they manage to make it through.
Just doing that and having solid, experienced programmers makes TDD close to obsolete in my opinion.
One book I can really recommend for any development really (except for maybe the chapter on TDD when it comes to games then, hehe) is Clean Code by Robert C. Martin. I don’t agree with some of it, like most people with most books, but It’s really quite funny and has a very beautiful vision when it comes to software engineering.
My favorite quote is probably:
Even though Clean code actually quotes that from another book XD (Grady Booch’s Object Oriented Analysis and Design with Applications).
Still the book is full with nice passages like that one.
I kind of went on some tangents here… sorry about that, but this is a subject we talk quite a lot about at my workplace, and I couldn’t help but gush :3
Thanks for the detailed reply - I was worried that the topic would be dismissed as not applicable to games. I’ll be following up those resources soon, too, so thanks a bunch!
What you’re saying totally makes sense when it comes to experimental programming, which a lot of games is. I’m working at an arch viz. and simulation company, and so it’s less about creativity for me and more trying to get specific systems to work properly and to not fall over, so I’d say that I’d benefit more from TDD than an average UE4 user - no gameplay here!
I think there are even some simple cases off of the top of my head for a game without as much planning - if you ended up with a base class common to most things, i.e. ‘AVehicle’ for a game involving driving all sorts of vehicles, you could unit test that it did very specific things under specific circumstances, no? I totally get what you’re saying, though.
Your point about keeping the architecture robust, and code well-structured; is TDD and its effects not a way to help achieve this? Makes it harder for bugs to get through, and forces everyone to really think about what method x and y should really be doing. I say all of this without much/any TDD experience, so take it was a grain of salt; this is just how I understood the idea.
I’ll check that book out, thank you. And don’t worry about the tangent - I really appreciated it! I’d happily talk through program architecture with you if you had the time - I’m always looking to improve that part of my programming skillset.
You are probably right that base classes might be better targets for TDD than leaf classes. The problem more in the way that these base classes are formed. Now if you already have decided upon a class hierarchy then it might work, but as the rest of the code, the base classes have a tendency to grow more organically as you identify similarities between new assets. Another problem is that base classes tend to have very simple methods. And the simpler the method, the less bang for the buck you will get for TDD (although the regression testing aspect will still be as great as ever of course :D).
Regarding the architecture, I agree that TDD is a very good way to get a good design, a class grown with TDD is much more likely to be well designed under SOLID principles than one that is not made using TDD from my experience, but this does not mean that it is not possible to create well designed classes without TDD of course.
In the end it just comes down to time investment concerns for me. I know that I can design a class without TDD that is (just grabbing a number out of my **** here) let’s say 90% as “good” as the version I would have come up with using TDD. But using TDD basically means I need to spend double the time creating code for the class, assuming I’m only doing one iteration. It is true that I will make some time “back” as one of the main benefits you get from TDD is less stupid bugs etc. but as soon as I need to start iterating, that time spent amending and adjusting my test code just becomes a burden.
In the end, I guess I see TDD as just another tool that has a very very specific usage scenario.
It does seem like it might actually be a good fit for you guys though, even though I would probably recommend maybe splitting your business logic (the stuff that you want to unit test) mostly into BP libraries or otherwise statically accessible classes since I’ve heard that unit testing functions on actors that require the world to be running or to be in a particular state is nightmare-inducing. It’s not really a unit test at that point either, but more of an integration test.
A compromise might be to contain business logic that you must have attached to actors as either POCO classes or UObject derived classes that you can create / mock to your hearts content in your tests, and then use those as UPROPERTY pointers in your actors. This is probably good design anyhow
The big downside is of course that testing anything that has to do with actor interactions or other interactions that would only occur in runtime will become hard. You could probably still simulate them, but it wouldn’t be an accurate simulation, and then what’s the point really?
I think a lot of people would love to hear how it went for you guys though regarding this if you decide to try it, as it is really quite interesting
I have been trying to figure out how to write SOLID code in UE4 and fully tested. I am working on a basic testing framework that I think will accomplish this.
I’m using a level for the testing map and UI widgets to display test results as they run. So I can see how many passed, failed, skipped or were marked incomplete.
I use base actor to build a simple test case class and test suite class. I then have a structure to represent the data provider elements. And use that to make data tables with my various datasets that include cases that should pass and a couple that should fail.
I’m working a way to mock a class next.
Just curious if you guys have tried similar approaches and what roadblocks you’ve ran into.
I never ended up getting anywhere with it - it was just too cumbersome to have to boot up UE4 to test, and then close down and restart (unit tests don’t hot-reload, by design apparently?).
Hello! I encountered a problem that prevents me from implementing my product. Everything is fine in the code, but there are still bugs when using it. How can I solve the problem?
@ I strongly disagree, one of the biggest plagues of the gaming industry at the moment is as a result of this thinking. Extreme crunches, low quality releases riddled with bugs, slow time-to-squash these bugs, and most importantly the rising early access & semi-finished release w/ live service model are all results of unprofessional software development as a result of untested code.
It is impractical to forego automation, games are no longer one-and-done. They are expected to be alive and updated with DLC and patches
Regarding experimentation: Unreal Engine has that for you in the form of Blueprints. You can prototype all you want using the tool for the job. No need for unit testing here, the Blueprints will not exist for long. Then once you have decided on a design (and you must decide on a design at one point or the game doesn’t get made), you start implementing the full feature in C++ with TDD in mind.
Unreal Engine needs a full web development or Java-style fleshed out unit testing, integration, and e2e frameworks that include all the necessary mocking, spying, and fixture functions.
Bumping this thread as a feature request to Epic! We need more automation & unit testing support to ship high-quality bug-free long-running changing games. Also a dev diary on how you test Fortnite and sustain the live service model without bug disasters.
Hi @NorthWeapon ,
We are currently working on a new framework for automated tests and will release this plugin on Github in a few weeks.
It will also support unit tests for blueprints.
Would you be interested in testing the plugin?
