Announcement

Collapse
No announcement yet.

In automated tests, how to check that an assertion is failing as expected ?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    In automated tests, how to check that an assertion is failing as expected ?

    I'm writing a C++ module to add support for ridiculously big numbers for an incremental game. I have a set of automated tests (using the new Automation Spec) to verify the behavior of arithmetic operations.

    I want to prevent anyone to accidentally divide by zero (and destroying the universe) so I'm crashing the engine if the divider is equal to zero, using a good old assertion.

    Code:
    verifyf(B.m != 0.f, TEXT("Cannot divide by 0"));
    As expected, the test below crashes the engine. The problem is that it prevents other tests to execute.

    Code:
            It("should not allow division by 0", [this]
            {
                FBigNumber A(2.f, 0);
                FBigNumber B(0.f, 0);
    
                FBigNumber result = A / B;
            });

    Is there a way to write an automated test that checks that the engine would have crashed, without actually crashing the engine?
    Co-founder and developer at Big Horn Studio

    #2
    Just wrap the verifies in a #define or some such.

    #if UE_BUILD_SHIPPING
    verifyf(B.m != 0.0f, TEXT("Cannot divide by 0"));
    #else
    Log(LogTest, Warning, TEXT("Cannot divide by 0"));
    #endif
    Able Ability System - A high performance, robust ability system for UE4. Now Available!

    Comment


      #3
      ensureMsg(expression, message);

      gives a call stack, but it shouldn't crash the engine, it keeps running.
      | Savior | USQLite | FSM | Object Pool | Sound Occlusion | Property Transfer | Magic Nodes | MORE |

      Comment


        #4
        Hi there,
        Thanks @ExtraLifeMatt and @BrUnO XaVIeR for your responses.
        I have exactly the same problem as @WirelessKiwi :
        I want to catch errors triggered by asserts only in the test environment, but I want the assert to crash the engine on other builds.
        So in regards of your suggests @ExtraLifeMatt, I manage to do something like this :

        Code:
        // -- In a dedicated file, an helper file
        #include "CoreMinimal.h"
        
        #if WITH_DEV_AUTOMATION_TESTS
        #define mycheck(expr)          \
            {                          \
                if (UNLIKELY(!(expr))) \
                {                      \
                    throw TEXT(#expr); \
                }                      \
            }
        #else
        #define mycheck(expr) check(expr)
        #endif
        
        // So now I can use mycheck(AnObject != nullptr) in my source code
        
        // -- in the .spec.cpp file
        try {
            // here the code which aimed to throw an error
            UMyClass AnObject = nullptr;
            mycheck(AnObject != nullptr);
            // this to ensure failing the test if the code above doesn't throw an error
           TestTrue("Should not be called", false);
        }
        catch (const TCHAR* e)
        {
            // just to get an output during test process
            UE_LOG(LogTemp, Display, TEXT("Exception occured for: %s"), e);
            TestEqual("Error has been triggered: AnObject != nullptr", e, TEXT("AnObject != nullptr"));
        }
        Do you think It's something reliable ? @WirelessKiwi did you found a better solution since?
        Last edited by nansP; 03-30-2020, 06:38 AM.

        Comment

        Working...
        X