In automated tests, how to check that an assertion is failing as expected ?

Development Programming & Scripting C++
1

I’m writing a C++ module to add support for ridiculously big numbers for an incremental game. I have a set of automated tests (using the new Automation Spec) to verify the behavior of arithmetic operations.

I want to prevent anyone to accidentally divide by zero (and destroying the universe) so I’m crashing the engine if the divider is equal to zero, using a good old assertion.


verifyf(B.m != 0.f, TEXT("Cannot divide by 0"));

As expected, the test below crashes the engine. The problem is that it prevents other tests to execute.



        It("should not allow division by 0", [this]
        {
            FBigNumber A(2.f, 0);
            FBigNumber B(0.f, 0);

            FBigNumber result = A / B;
        });

Is there a way to write an automated test that checks that the engine would have crashed, without actually crashing the engine?

2

Just wrap the verifies in a #define or some such.

3

ensureMsg(expression, message);

gives a call stack, but it shouldn’t crash the engine, it keeps running.

4

Hi there,
Thanks @**ExtraLifeMatt and @BrUnO XaVIeR for your responses.
I have exactly the same problem as @WirelessKiwi : **
I want to catch errors triggered by asserts **only in the test environment, but I want the assert to crash the engine on other builds.
So in regards of your suggests @ExtraLifeMatt, **I manage to do something like this :



// -- In a dedicated file, an helper file
#include "CoreMinimal.h"

#if WITH_DEV_AUTOMATION_TESTS
#define mycheck(expr)          \
    {                          \
        if (UNLIKELY(!(expr))) \
        {                      \
            throw TEXT(#expr); \
        }                      \
    }
#else
#define mycheck(expr) check(expr)
#endif

// So now I can use mycheck(AnObject != nullptr) in my source code

// -- in the .spec.cpp file
try {
    // here the code which aimed to throw an error
    UMyClass AnObject = nullptr;
    mycheck(AnObject != nullptr);
    // this to ensure failing the test if the code above doesn't throw an error
   TestTrue("Should not be called", false);
}
catch (const TCHAR* e)
{
    // just to get an output during test process
    UE_LOG(LogTemp, Display, TEXT("Exception occured for: %s"), e);
    TestEqual("Error has been triggered: AnObject != nullptr", e, TEXT("AnObject != nullptr"));
}

Do you think It’s something reliable ? @**WirelessKiwi **did you found a better solution since?

5

any news on how to achieve this?

6

I solved this with the next structure

#if WITH_DEV_AUTOMATION_TESTS && !UE_BUILD_SHIPPING
	if (!FAutomationTestFramework::GetInstance().GetCurrentTest())
	{
		check(YourCondition);
	}
#endif

but would love a fancier way for this tbh

7

You could wrap the condition in a separate function and then explicitly test that function with different inputs.