Hello all! Does the Unreal Engine team have any plans to write automated tests for Unreal Engine? I think this would provide a lot of value across the entire gaming industry, since Unreal Engine is used by so many game development teams. So many games are infamous for “glitches” or bugs which could have been prevented by having the right testing frameworks in place. This includes both C++ code and Blueprints.
There is a simple test automation framework within Unreal Engine, but it doesn’t seem to scale to the entire Unreal Engine codebase. This will likely need to be replaced with a full testing framework to build unit tests for the entire Unreal Engine codebase.
In my experience as a software developer, unit tests often catch bugs before the code is even released. Every unit test permanently prevents a bug, instead of temporarily solving the bug until the code changes again. Unreal Engine is limited in how fast it can grow due to the lack of test automation. Manual QA testing is a lot more expensive than an automated test, since it consumes human resources and takes much more time. Manual QA is still great for finding new bugs or manually running integrated tests that would be hard to write automated integration tests for. In addition, writing unit tests for Unreal Engine will allow customers to write unit tests for their Unreal Engine plugins and games.
Writing automated tests for Unreal Engine will require up front investment to set up testing as part of the Unreal Engine Build Tool, to integrate automated tests with the generated IDE projects, and to extend or build a test framework which integrates with the core components of Unreal Engine. After this, each component in Unreal Engine can start to write unit tests. This will involve refactoring the existing codebase to use C++ pure virtual classes to access dependencies in a way which can be mocked. In addition, we need to build testing components for Blueprints to allow gameplay to be automatically tested by Blueprint customers.
Does the Unreal Engine team feel that Unreal Engine bugs are enough of a problem to justify this investment? How does Unreal Engine prevent bugs today? Is integration testing more of a problem than unit testing?
How would the Unreal Engine team feel about accepting pull requests related to setting up unit testing and writing some initial tests? The goal of these contributions would be to provide concrete data showing how the Unreal Engine team would the benefit from unit testing.