In 4.11 we added a dropdown to the frontend that lets you select which collection of tests is displayed. These filters are determined by the EAutomationTestFlags::*Filter options. Since yours uses the Smoke filter, you’d want to choose Smoke Tests in the dropdown.
We recommend that you don’t use the ApplicationContextMask flag to mark which application types you want to test. Instead, it’s better to explicitly combine the flags for the applications you want.
Also, marking a test with the SmokeFilter flag will cause it to run every time the application runs (except in Shipping builds). This is fine for quick tests that must always work (eg. do basic core engine systems function?), but is not recommended for most tests. It will also automatically fail if the tests takes more than a second or two to ensure that all combined smoke tests can run in a reasonable amount of time during startup. If you’re testing something in your game, you most likely want the ProductFilter flag.
Ha right. I missed that it was a Complex test. For one-off tests, you want IMPLEMENT_SIMPLE_AUTOMATION_TEST. You only need Complex if you’re creating parameterized test cases.