Announcement

Collapse
No announcement yet.

Automation Testing Should be on the Roadmap

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • replied
    Originally posted by alpaka View Post
    ... Has anyone experimented with writing simulation code in unmanaged parts of the game (POCOs) and using the VS test toolkit?
    To answer my own earlier question: If you are fine with (or even prefer) foregoing all the UE4 C++ extensions by using POCOs, it is quite possible to create a library project, which you can unit test with the regular VS test system, then link it into your project. This allows you to use unmanaged code both in the test explorer and in your game. If you need to unit test UObject code, need reflection, or are using managed code otherwise, I'm afraid you're stuck with the built-in testing framework.

    There are basically two options: a static library (.lib) or dynamic library (.dll). I couldn't easily figure out how to use the hot reload feature for non-module DLLs, or how to build them as stand-alone DLLs (can't link to the generated DLLs from the build tool, it seems), so I went with the static library. There is an introduction to linking them using UBT here: https://wiki.unrealengine.com/Linkin...e_Build_System, but you should update it to 4.16 using ReadOnlyTargetRules instead of TargetInfo.

    I would also like to note the following workaround for using the automation system a little more efficiently (quoting myself on the Answer hub):
    Just in case anyone is looking for a sorta workaround to this: I played around a bit with hot reloading modules that contain tests and while you cannot change an existing test, adding a new one seems to work fine (recompile module from the editor - or, if the test is in your main module, it also works when building from VS - then hit Refresh Tests in the session frontend).

    So when you want to change an existing test to make it pass or whatever, you can change its name from Test to something else (I use Test2, Test3, etc.). Unfortunately, the old test class stays around as a ghost until you restart the editor, so it messes up the "Run All tests in a category" functionality. Note that you can have multiple test classes in a single source file so genuinely adding new tests is not a big issue.

    While this is obviously not a perfect solution, it's somewhat workable and is at least better than having to restart the editor all the time. Once in a while, you can clean up all the numbers and just restart the editor.

    Leave a comment:


  • replied
    I kind of think that those people who seriously use this sort of thing would probably pay for a license.

    I think it is a bit sneaky as well and has a bit of wisdom. I've being in work places where devs (having no budget on the own) just start using these sorts of tools on the sly. Along comes some compliance person (or busy body, whatever) and kind of forces the issue (which if the devs went through the "proper process" would just being blown off - sometimes it is easier for the money person just to say yes).

    Leave a comment:


  • replied
    Originally posted by funkinessfactor View Post
    I just followed this...

    Their blog post about using PVS Studio for free.

    https://www.viva64.com/en/b/0457/
    Now that's an interesting license... they're basically asking us to write some kind of automation script to add/remove those comments

    Leave a comment:


  • replied
    I just followed this...

    Their blog post about using PVS Studio for free.

    https://www.viva64.com/en/b/0457/

    Leave a comment:


  • replied
    Originally posted by funkinessfactor View Post
    and us single independent developers working on individual independent projects get it for free.
    Where/how did you get PVS studio for free?

    Leave a comment:


  • replied
    Obviously there are many AAA studios that use Unreal. Maybe Epic could reach out and ask what they do on this front and provide suggestions to the great unwashed. Maybe even a guest blog post. Hopefully they don't come back saying they employee testers @ $2 per hour

    For myself, I have heaps of checks in my code and a few check(TestWorldConsitency()); type things. I have a few "test maps" that automate stuff on Tick(), but it is really unrefined. Please don't flame me to say but you are not doing real automated tests or unit tests.

    I've also got some millage from using the PVS-Studio which you can see that better support is on the roadmap for 4.17. I'm sure I have a relatively small project, but it only took me a day to get everything passed (or marked as a false positive) and us single independent developers working on individual independent projects get it for free.

    Leave a comment:


  • replied
    +1

    Really a sad state of affairs as far as unit testing is concerned. It would seem to me as if unit testing would be quite useful for many of the more math-heavy parts of code that one develops (such as AI or simulation code).

    Having to jump through absurd hoops to even get a single test to run (took me more than an hour, especially because all docs you can easily find are outdated and you have to go with your gut/experiment quite a bit), let alone lacking support for hot-plugging, etc. is so painful that it makes skipping unit tests seem a very attractive option. Has anyone experimented with writing simulation code in unmanaged parts of the game (POCOs) and using the VS test toolkit?

    Leave a comment:


  • replied
    Bump ! Any news about this "issue" ?

    Unit Testing is a must in Software Engineering... and Unit tests deserve some Love
    Last edited by Cenovis; 06-14-2017, 04:26 AM.

    Leave a comment:


  • replied
    Are there any more updates on this?

    Would really be a massive help to have a proper documentation Automation System and proper unit tests.

    Leave a comment:


  • replied
    Hi guys,

    As i´m coming from a enterprise java background and love to write my game in spare time, testing with UE4 is really hard to achieve to be honest.
    Is there any update on that topic? Maybe it is a reachable goal to have some proper testing framework like junit + mockito from java in the UE4 world, nobody says it needs to be 100%, but usually a proper optimized workflow would help.

    I´m not that experience in c++ right now, but i think if we work all together, this is a possible thing!

    Leave a comment:


  • replied
    Hi, I've taken your feedback here and escalated it to the appropriate teams for review. Thanks for giving us your feedback and I'm sorry for any headaches this has caused

    Leave a comment:


  • replied
    Yeah, not having testing seems like a big oversight. This is 2016. A test suite should be a given.

    Leave a comment:


  • replied
    Oof. I'm trying to assess some of the "lesser talked about features" in UE4 and this one stood out as a biggie. We tend to write turn-based strategy games, so a lot of game logic tends to be written in testable form. The front end all looks great, and the tools seem nice, but man... lacking hot-reload is a huge time suck. If anyone has any work-arounds via external tools or other strategies to writing unit/integration tests in UE4 I'd love to hear it.

    Leave a comment:


  • replied
    I strongly agree that we need to be able to hot-reload the tests. TDD and Automated testing are how modern software is made.
    Please make automation a first-class feature for game developers.

    Leave a comment:


  • replied
    Bad news, friends - automation tests not hot-reloading is not a bug. Still hoping for a change so we can actually use this framework for quick and iterative project-level testing.

    Leave a comment:

Working...
X