UnrealBuildTool, SubmitTool: SubmitTool's default `NonUnityAndStaticAnalysis` validator fails when an Engine source file is checked out by trying to include `Engine\Restricted\NotForLicensees\Binaries\Win64\AutoRTFM`

Hey folks,

I’m trying to set up SubmitTool as part of my studio’s Perforce submit workflow, and an issue was raised by one of our members who is making changes to the Engine source. For context, this is in a Source engine build with a native project. We have not yet made our own SubmitTool.ini, so it’s all default settings for SubmitTool. We’re on the GitHub sources for Unreal Engine version 5.7.4, commit 260bb2e1c5610b31c63a36206eedd289409c5f11.

When running SubmitTool’s default validator, NonUnityAndStaticAnalysis, on a changelist with a checked out Engine source file, it will always fail with this message:

Error: [Non Unity + SA Validator (#1)]: Unhandled exception: DirectoryNotFoundException: Could not find a part of the path '%P4Workspace%\Engine\Restricted\NotForLicensees\Binaries\Win64\AutoRTFM'.
Full log and target lists

Log:

LogValidators: [Non Unity + SA Validator] Using Target: Win64 Development -TargetType=Editor -Define:CHECK_PUREVIRTUALS=1 -Define:UE_CHECK_DISABLE_OPTIMIZATION=1 -NoPCH -DisableUnity -AllModules -NoLink -WarningsAsErrors -IncludeHeaders -staticanalyzer=Default -FileList="%LocalAppData%/UnrealGameSync/Tools/SubmitTool/Current/Windows/Engine/Intermediate/SubmitTool/FileLists/5A1C01EA-4FAB-58C4-54FA-E18780E8A407.txt"
LogValidators: [Non Unity + SA Validator (#1)]: Running process #1: %P4Workspace%/Engine/Build/BatchFiles/RunUBT.bat -TargetList="%LocalAppData%/UnrealGameSync/Tools/SubmitTool/Current/Windows/Engine/Intermediate/SubmitTool/TargetLists/227918BB-4130-1064-C3AB-96B2A31238F5.txt"
LogValidators: [Non Unity + SA Validator (#1)]: Task process started.
LogValidators: [Non Unity + SA Validator (#1)]: Using bundled DotNet SDK version: 8.0.412 win-x64
LogValidators: [Non Unity + SA Validator (#1)]: When looking for per-project SDK overrides, we got a commandline that could not be used to find a target project (Win64 Development -TargetType=Editor -Define:CHECK_PUREVIRTUALS=1 -Define:UE_CHECK_DISABLE_OPTIMIZATION=1 -NoPCH -DisableUnity -AllModules -NoLink -WarningsAsErrors -IncludeHeaders -staticanalyzer=Default -FileList=%LocalAppData%/UnrealGameSync/Tools/SubmitTool/Current/Windows/Engine/Intermediate/SubmitTool/FileLists/5A1C01EA-4FAB-58C4-54FA-E18780E8A407.txt)
LogValidators: [Non Unity + SA Validator (#1)]: Log file: %P4Workspace%\Engine\Programs\UnrealBuildTool\Log.txt
LogValidators: Error: [Non Unity + SA Validator (#1)]: Unhandled exception: DirectoryNotFoundException: Could not find a part of the path '%P4Workspace%\Engine\Restricted\NotForLicensees\Binaries\Win64\AutoRTFM'.
LogValidators: [Non Unity + SA Validator (#1)]:    at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
LogValidators: [Non Unity + SA Validator (#1)]:    at System.IO.Enumeration.FileSystemEnumerator`1.Init()
LogValidators: [Non Unity + SA Validator (#1)]:    at System.IO.Enumeration.FileSystemEnumerable`1..ctor(String directory, FindTransform transform, EnumerationOptions options, Boolean isNormalized)
LogValidators: [Non Unity + SA Validator (#1)]:    at System.IO.Enumeration.FileSystemEnumerableFactory.UserDirectories(String directory, String expression, EnumerationOptions options)
LogValidators: [Non Unity + SA Validator (#1)]:    at System.IO.Directory.InternalEnumeratePaths(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
LogValidators: [Non Unity + SA Validator (#1)]:    at EpicGames.Core.DirectoryReference.EnumerateDirectories(DirectoryReference baseDir) in %P4Workspace%\Engine\Source\Programs\Shared\EpicGames.Core\DirectoryReference.cs:line 370
LogValidators: [Non Unity + SA Validator (#1)]:    at UnrealBuildTool.MicrosoftPlatformSDK.<>c__DisplayClass27_0.<FindToolChainInstallations>b__0(WindowsCompiler _) in %P4Workspace%\Engine\Source\Programs\UnrealBuildTool\Platform\Windows\MicrosoftPlatformSDK.cs:line 784
LogValidators: [Non Unity + SA Validator (#1)]:    at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
LogValidators: [Non Unity + SA Validator (#1)]:    at UnrealBuildTool.MicrosoftPlatformSDK.TryGetToolChainDir(WindowsCompiler Compiler, String CompilerVersion, UnrealArch Architecture, ILogger Logger, VersionNumber& OutToolChainVersion, DirectoryReference& OutToolChainDir, DirectoryReference& OutRedistDir) in %P4Workspace%\Engine\Source\Programs\UnrealBuildTool\Platform\Windows\MicrosoftPlatformSDK.cs:line 552
LogValidators: [Non Unity + SA Validator (#1)]:    at UnrealBuildTool.VCEnvironmentParameters..ctor(WindowsCompiler Compiler, WindowsCompiler ToolChain, UnrealTargetPlatform Platform, UnrealArch Architecture, String CompilerVersion, String ToolchainVersion, String WindowsSdkVersion, String SuppliedSdkDirectoryForVersion, Boolean bUseCPPWinRT, Boolean bAllowClangLinker, Boolean bAllowRadLinker, ILogger Logger) in %P4Workspace%\Engine\Source\Programs\UnrealBuildTool\Platform\Windows\VCEnvironment.cs:line 502
LogValidators: [Non Unity + SA Validator (#1)]:    at UnrealBuildTool.WindowsPlatform.CreateVCEnvironment(TargetRules Target) in %P4Workspace%\Engine\Source\Programs\UnrealBuildTool\Platform\Windows\UEBuildWindows.cs:line 1232
LogValidators: [Non Unity + SA Validator (#1)]:    at UnrealBuildTool.WindowsPlatform.ValidateTarget(TargetRules Target) in %P4Workspace%\Engine\Source\Programs\UnrealBuildTool\Platform\Windows\UEBuildWindows.cs:line 1405
LogValidators: [Non Unity + SA Validator (#1)]:    at UnrealBuildTool.RulesAssembly.CreateTargetRulesInstance(String TypeName, TargetInfo TargetInfo, ILogger Logger, Boolean IsTestTarget, TargetRulesValidationOptions ValidationOptions) in %P4Workspace%\Engine\Source\Programs\UnrealBuildTool\System\RulesAssembly.cs:line 804
LogValidators: [Non Unity + SA Validator (#1)]:    at UnrealBuildTool.RulesAssembly.GetTargetNameByType(TargetType Type, UnrealTargetPlatform Platform, UnrealTargetConfiguration Configuration, FileReference ProjectFile, ILogger Logger) in %P4Workspace%\Engine\Source\Programs\UnrealBuildTool\System\RulesAssembly.cs:line 944
LogValidators: [Non Unity + SA Validator (#1)]:    at UnrealBuildTool.TargetDescriptor.ParseSingleCommandLine(CommandLineArguments Arguments, Boolean bUsePrecompiled, Boolean bSkipRulesCompile, Boolean bForceRulesCompile, Boolean bCreateSimplifiedDescriptors, List`1 TargetDescriptors, ILogger Logger) in %P4Workspace%\Engine\Source\Programs\UnrealBuildTool\Configuration\TargetDescriptor.cs:line 545
LogValidators: [Non Unity + SA Validator (#1)]:    at UnrealBuildTool.TargetDescriptor.ParseCommandLine(CommandLineArguments Arguments, Boolean bUsePrecompiled, Boolean bSkipRulesCompile, Boolean bForceRulesCompile, Boolean bCreateSimplifiedDescriptors, List`1 TargetDescriptors, ILogger Logger) in %P4Workspace%\Engine\Source\Programs\UnrealBuildTool\Configuration\TargetDescriptor.cs:line 380
LogValidators: [Non Unity + SA Validator (#1)]:    at UnrealBuildTool.TargetDescriptor.ParseCommandLine(CommandLineArguments Arguments, Boolean bUsePrecompiled, Boolean bSkipRulesCompile, Boolean bForceRulesCompile, Boolean bCreateSimplifiedDescriptors, List`1 TargetDescriptors, ILogger Logger) in %P4Workspace%\Engine\Source\Programs\UnrealBuildTool\Configuration\TargetDescriptor.cs:line 365
LogValidators: [Non Unity + SA Validator (#1)]:    at UnrealBuildTool.TargetDescriptor.ParseCommandLine(CommandLineArguments Arguments, BuildConfiguration BuildConfiguration, ILogger Logger) in %P4Workspace%\Engine\Source\Programs\UnrealBuildTool\Configuration\TargetDescriptor.cs:line 284
LogValidators: [Non Unity + SA Validator (#1)]:    at UnrealBuildTool.BuildMode.ExecuteAsync(CommandLineArguments Arguments, ILogger Logger) in %P4Workspace%\Engine\Source\Programs\UnrealBuildTool\Modes\BuildMode.cs:line 176
LogValidators: [Non Unity + SA Validator (#1)]:    at UnrealBuildTool.UnrealBuildTool.Main(String[] ArgumentsArray) in %P4Workspace%\Engine\Source\Programs\UnrealBuildTool\UnrealBuildTool.cs:line 653
LogValidators: [Non Unity + SA Validator (#1)]: Result: Failed (OtherCompilationError)
LogValidators: [Non Unity + SA Validator (#1)]: Total execution time: 0.87 seconds
LogValidators: Error: [Non Unity + SA Validator (#1)]: RunUBT ERROR: UnrealBuildTool exited with a failure
LogValidators: [Non Unity + SA Validator (#1)]: Completed running process #1. Process took 1.80 sec and exited with code 6
LogValidators: Error: [Non Unity + SA Validator (#1)]: Task process failed with exit code 6 and 2 log errors.
LogValidators: Error: ========================[Errors Summary #2]========================
LogValidators: Error: [Non Unity + SA Validator (#1)]: Unhandled exception: DirectoryNotFoundException: Could not find a part of the path '%P4Workspace%\Engine\Restricted\NotForLicensees\Binaries\Win64\AutoRTFM'.
LogValidators: Error: [Non Unity + SA Validator (#1)]: RunUBT ERROR: UnrealBuildTool exited with a failure
LogValidators: Error: [Non Unity + SA Validator (#1)]: Task process failed with exit code 6 and 2 log errors.
LogValidators: Error: ================================================================

SubmitTool/TargetLists/227918BB-4130-1064-C3AB-96B2A31238F5.txt:

Win64 Development -TargetType=Editor -Define:CHECK_PUREVIRTUALS=1 -Define:UE_CHECK_DISABLE_OPTIMIZATION=1 -NoPCH -DisableUnity -AllModules -NoLink -WarningsAsErrors -IncludeHeaders -staticanalyzer=Default -FileList="%LocalAppData%/UnrealGameSync/Tools/SubmitTool/Current/Windows/Engine/Intermediate/SubmitTool/FileLists/5A1C01EA-4FAB-58C4-54FA-E18780E8A407.txt"

SubmitTool/FileLists/5A1C01EA-4FAB-58C4-54FA-E18780E8A407.txt:

%P4Workspace%/Engine/Source/Runtime/Engine/Private/Materials/HLSLMaterialTranslator.cpp

The validator will only work when the changelist only contains files from %P4Workspace%/%ProjectName%/.... The error will still be thrown if the changelist has checked out files in both the Engine/... directory and the %ProjectName%/... directory.

To track down where this AutoRTFM requirement was being added, I tried breakpoint debugging UnrealBuildTool using the stack trace in the full log, but I was only able to ascertain that the requirement was coming from a different code path than the one in the stack trace.

I did some internet searching and in this forum, but haven’t seen any reports of this issue. Has anyone else encountered it?

Hi Willem, exciting to see someone from Red Hook posting in the UE forums! I’m a fan of Darkest Dungeon 1 and 2 myself. :slight_smile:

I apologize that this issue is affecting you, and I can confirm that this is a problem with Unreal Engine 5.7 releases. The simplest way to work around this issue is to delete the directory Engine/Source/Programs/AutoRTFMTests and all of its contents from your copy of Unreal Engine. The way SubmitTool invokes UBT is causing this target to be constructed which then throws the exception you’re experiencing, and this target isn’t meaningfully usable by licensees, so it’s safe to remove, and this should fix your problem.