We’re aware that there’s already some code in place for running functional tests in Unreal, but we think that the engine is still lacking in that area with respect to both documentation and functionality. Following the open-source spirit of the engine, we’d like to share with you, free of charge:
43 assertions
6 custom delay nodes (e.g. wait until trigger box was triggered)
input simulation
test trigger boxes
timeouts
lifecycle events (e.g. BeforeAll, BeforeEach)
parameterized tests
skipping tests (temporarily)
assumptions (e.g. runtime platform)
We’re using the plugin extensively for *The Lord of the Rings™: Gollum™, *and would like to share our current results with you, hoping that it will help you and that maybe, we can work together on improving it instead of having everyone “roll their own” when it comes to test automation.
In case you’re interested in “why you should care”, more details about how to create and run tests as well as other of the above features, feel free to check out the official documentation. If many people are interested, we might as well add it to the Unreal Marketplace later ™.
As always, any feedback (or even contributions) are highly appreciated!
Give these humans a medal. Great work.
I haven’t seen anything except an unreviewed automation plugin on the marketplace and a plugin for selenium. I was about to hijack selenium 4 beta and make a fork for unreal and call it unrealium but make everything work off of hypothesis (https://hypothesis.works/) and have it generate tests for us. but then you all just released this and idk if its even worth it and I have no plans on going forward with any more. That said, **Has anyone on the team looked into modern implementations of property based testing (QuickCheck)? **
Really I don’t think many people will appreciate or understand the importance of testing here, maybe its changed but it seems there’s a lack of interest in software in general and not just games. I was a Technical Consultant for the past two years and focused in testing. Primarily I wrote a compiler, interpreter, and REPL for a programming language that makes non-coders (business associates) write the automation for website testing in plain english. Making people understand “why they should care” about testing is almost impossible for people set in their ways, which is why I recently left. The reason I ask about things like quick check is we’ve had the tools now to make test writing automated for some time. With what you guys have released, I think this would actually be a really good basis for it. I found even with automated automation, people were still like, why should we care though…
Can you provide links to the unreviewed automation plugin and to the Selenium plugin? I was unable to find both of them on the marketplace.
Hypothesis and QuickCheck look promising. Our test automation plugin supports parametrized tests, even with what we call dynamic “parameter providers”, so it would be pretty easy to hook up either Hypothesis or QuickCheck with it (in case you get them running with UE4). We haven’t had the need for that (yet), as we’re looking out more for the integration side of testing - unit tests already look pretty fine to us from what Unreal Engine already provides.
For me, “making people care” was a big topic earlier this week, when I presented the results internally. In our case, it helped to illustrate both
the test burden that comes with the increasing number of “dimensions” to test (e.g. in our case making a multi-platform game with a relatively small team)
the amount of time that can be spent on other tasks while automation takes care of testing
That’s very good news. Adding a QuickCheck would definitely be the next thing to do in my opinion. I will look at some QuickCheck implementations and see if its not too much work to get it in a plugin itself. As for the integration side of testing, that’s probably where I would have started it off too. since, like you said unreals unit tests are okay for the most part.
Wow, the first one would seem like an obvious reason, but for a few years at the company i worked for none of us could see the forest through the trees. We worked on some of the biggest, shittiest sites on the internet. So we had lots and lots of teams on just a single application on a single site for a client. Basically better testing equates to more clients because you don’t need as many heads working on a client. I feel super dumb for not seeing this.
