I’m trying to implement simple test im Unreal Engine 5, but I stuck on creating an empty map.
The following code crashes with this error:
[2022.05.09-17.50.13:609][519]LogAutomationController: Error: EditCondition attempted to use an invalid operand "WorldPartition".
[2022.05.09-17.50.13:609][519]LogAutomationController: Error: EditCondition attempted to use an invalid operand "TextureSizingType" (value error).
I also tried to open existing map, and some other approaches of empty map creating, but all of them fail with this error.
I also found engine tests, that uses this function, like FLandscapeEditorTest and it also fails.
Is it a known bug in Unreal Engine, or I configure smth wrong?
BTW: I tried both linux build and windows epic games build. Nothing helped.
UPD: It seems that map is created succesfully, and test finishes, but as somewhere in editor error is thrown, test is marked as “aborted”.
Bumping this because I’m having the same errors while running automated tests in UE5. This is not a new map in my case, as the map was created prior to switching engine versions to UE5.
Make sure you set in your Editor Settings the Automation Testmap to be an empty map in your project. For example I have Maps/L_TestAutomation in my project and I have it configured as the Automation Testmap. This is a completely empty map with nothing at all in it.
Before I configured that I was getting errors similar to what you are experiencing.
If that doesn’t help, it may be your UE version. In 5.0.0 it would always give me errors the first time I tried to run tests after starting the editor. Ignoring those and simply running the tests again would succeed. The errors were similar to what you’re seeing now. As of 5.0.1 it’s working great with no errors even at startup, but it still seems to require that empty map setup in the preferences.
Yeah, thanks a lot @xi57. It fixed test run with Session Frontend in the editor. However, if I run it from terminal, or Rider, the issue is still here. @BrittanyRoseArt can you please share how you run your tests?
Well, if I now run the same test twice. Second attempt works.
It looks like the second issue is in editor startup warnings. It looks like @xi57 also experienced it in 5.0.0 trying to run tests for the first time. But I also have it in 5.0.1.
Well, it looks like at least I have a workaround for this.
class FMyAutomationTest : public FAutomationTestBase {
public:
FMyAutomationTest(const FString &InName, bool bInComplexTask)
: FAutomationTestBase(InName, bInComplexTask) {
}
virtual void AddError(const FString &InError, int32 StackOffset) override {
if (InError.Contains("will be marked as failing due to errors being logged")) {
return;
}
if (InError.Contains("EditCondition attempted to use an invalid operand")) {
return;
}
FAutomationTestBase::AddError(InError, StackOffset);
}
};
It’s possible the WorldPartition errors have to do with the map that you have loaded when you first try to run your tests.
At the moment the map I have loaded by default is super simple.
If you have a UE5 map with a WorldPartition as your default, that could be the reason you get errors the first time you run your tests, but not on any subsequent times.
If possible, try to swap your map before you run tests for the first time. Make sure it’s a UE4 vanilla empty map.
As for your question about running tests from the command line, I haven’t tried that (I’m a UE noob honestly) but maybe you can pass some parameter like -StartMap L_AutomationTestmap so that it’s loading up the automation test map rather than whatever the default editor map is? Just a wild guess.
I’m using just empty map, so it doesn’t seem it has World Partition. Anyway, I think I’m pretty happy with my workaround for a moment, so I could just wait for epics to fix this bug in the engine
Anyway, thanks a lot @xi57 I’ve spent almost 2 days trying to make tests work and almost gave up.
Setting the test map has done nothing for me either. I’ve done some digging in order to figure out why this might be happening and it’s quite a rabbit hole.
First of all, I actually don’t know why setting that map would work in the first place. UAutomationTestSettings::AutomationTestmap is only used in a handful of places in-engine, where the test explicitly loads that map rather than creating a new one. (See FStatsVerificationMapTest::RunTest for an example.) I’ve done some testing myself by placing an object in the test map and attempting to query for it during a test, and that query comes back empty, indicating that the map isn’t getting loaded when using FAutomationEditorCommonUtils::CreateNewMap. If this is working for you, then that’s fantastic, but I have zero idea why that would be the case.
The specific bug seems to be that an editor details customization for the world settings, FLightmapCustomNodeBuilder, is attempting to rebuild itself as part of handling FEditorDelegates::MapChange, but that customization is actually a stale one left over from the previous map. Rebuilding it walks down the tree until it starts trying to evaluate EditCondition metaproperties, at which point it realizes that its world settings pointer is stale and unhelpfully informs us about an invalid operand. This customization is deconstructed shortly thereafter.
This has nothing to do with WorldPartition. I’ve inserted other properties into my world settings object with meta=(EditCondition = "foo == nullptr") and those cause tests to fail as well.
The fact that this only causes tests to fail once is another quirk of the engine. It appears that these operand errors are only ever written to the log once, even though they fire every single time. I don’t know if this is intended behavior, and if it isn’t then this could change with any patch, so I would not rely on this behavior to solve the problem.
I think the best solution at the moment is to use the hack @Fexolm posted and just suppress the errors. I don’t like doing this (especially because the “will be marked as failing due to errors being logged” is a useful error to capture) but until Epic resolves this there’s little else we can do unless we want to start rooting around in how editor customizations are spun up when switching maps.
I’m having this issue with UE 5.3.2
Tried setting up the Test Automation Map in editor settings (in the editor).
Tried adding the workaround, but it doesn’t help.
Happens only at start and only for the test marked with: EAutomationTestFlags::SmokeFilter
so when I replaced it with: EAutomationTestFlags::ProductFilter
it finally worked.
Still it doesn’t look ok to me, as access violation doesn’t look like a reasonable feedback even if CreateNewMap() can’t be used with SmokeFilter.
If anyone else is having an issue writing automation tests where they rely on code which inherently throws errors by design and there is no workaround, here’s what worked for me.
Modify your DefaultEngine.ini file to include the following:
