Running ResavePackages Commandlet with -fixupredirects is not picking up all redirectors in our project

Development Pipeline & Plugins
, ,
1

We are attempting to automate a fixup redirector job like so in Build Graph:

<Commandlet Name="ResavePackages" Arguments="-fixupredirects -autocheckout -projectonly -ini:Engine:[Engine.ErrorHandling]:bCommandletWarningsAsErrors=False -skipcheckedoutpackages -KeepRedirectorDays=5 -Verbose" Project="<ProjectName>" UsePerforce="false"/>

Which eventually gets translated to this command:

UnrealEditor-Cmd.exe "<RootDir>\Projects\<ProjectName>\<ProjectName>.uproject" -run=ResavePackages -fixupredirects -autocheckout -projectonly -ini:Engine:[Engine.ErrorHandling]:bCommandletWarningsAsErrors=False -skipcheckedoutpackages -KeepRedirectorDays=5

The issue we’re seeing is that this command doesn’t seem to pickup all the redirectors that need to be fixed. After doing some investigation, we eventually root caused it to these lines in ContentCommandlets.cpp:

if (RedirectPackages.Contains(PackageName))
{
    RedirectorsToFixup.Add(PackageName);
}

Here, PackageName can contain absolute paths whereas RedirectPackages can only contain relative paths, so .Contains would return false and those packages would never get marked for fixup.

For example, one of our redirectors shows like this:

In RedirectPackages: “../../../Projects/<ProjectName>/Plugins/GameFeatures/Proto_Skins/Content/Gameplay/Struct_EffectOverride.uasset”

In PackageName: “T:/<RootDir>/Projects/<ProjectName>/Plugins/GameFeatures/Proto_Skins/Content/Gameplay/Struct_EffectOverride.uasset”

Only content in our GFPs located in \Plugins\GameFeatures\ seem to be affected, and the commandlet is able to find and fix redirectors located elsewhere just fine.

Is there something we’re doing wrong in our command that’s causing this issue, or does the above code block simply need to be updated to account for both relative + absolute paths?

[Attachment Removed]

2

Hello!

That could be a bug with GFPs and the commandlet.

Are the project files in RedirectPackages absolute paths? If so, the solution would be to convert the relative paths from GFPs to Absolute paths.

FString Abs = FPaths::ConvertRelativePathToFull(FPaths::ProjectDir(), Rel);

If that doesn’t work, I would require some data while I’m looking for a project I can use to test locally. What does RedirectPackages and PackageNames? I’m guessing that PackageNames is filled using NormalizePackageNames (line 337-338). It this the case?

Regards,

Martin

[Attachment Removed]

3

I reviewed you finding on passing the uproject path vs the project name. That would be related to the way the %GAMEDIR% variable is resolved. The code is in FGenericPlatformMisc::ProjectDir and there are multiple scenario to support.

I think the conversion to absolute paths is the best solution here. That guarantees that the “right” names are used and prevent issues if part of paths are the same.

[Attachment Removed]

4

RedirectPackages is all relative paths, and PackageName is a mix of relative and absolute paths.

PackageNames is indeed fetched from NormalizePackageNames, and RedirectPackages is fetched using FPackageName::SearchForPackageOnDisk

I was able to get this working locally by converting everything to absolute path before the comparison

FPackageName::SearchForPackageOnDisk(*AssetData.PackageName.ToString(), nullptr, &RedirectFile);
RedirectPackages.Add(FPaths::ConvertRelativePathToFull(RedirectFile), &bIsAlreadyInSet);

if (RedirectPackages.Contains(FPaths::ConvertRelativePathToFull(PackageName)))
{
    RedirectorsToFixup.Add(PackageName);
}

Would be happy to open a PR, if we’re OK with this fix :grinning_face:

[Attachment Removed]

5

I’d also add that changing the project argument passed into UnrealEditor-Cmd from the .uproject to just the project name also seems to fix the issue:

e.g. Instead of the above command, running this seems to make everything in PackageName use relative paths correctly:

UnrealEditor-Cmd.exe <ProjectName> -run=ResavePackages -fixupredirects -autocheckout -projectonly -ini:Engine:[Engine.ErrorHandling]:bCommandletWarningsAsErrors=False -skipcheckedoutpackages -KeepRedirectorDays=5The main difference is that this line in PackageUtilities.cpp:122 can return absolute paths if the .uproject is passed in, but does not if only the project name is passed in

if ( GConfig->GetArray( TEXT("Core.System"), TEXT("Paths"), Paths, GEngineIni ) > 0 )

[Attachment Removed]

6

Thanks! I was able to fix this locally by swapping to absolute paths so we’ll go with that for now to unblock our CI job.

It does still seem like there’s an engine issue though - the paths for content in GFPs should be returning the same relative paths regardless of whether the uproject path or project name is passed in right? Is that perhaps something we could get help with investigation on and fixed? This only causes issues with this commandlet right now, but I imagine the path inconsistency could cause all sorts of issues in GFPs in the future

[Attachment Removed]

7

Using absolute paths is the proper way to handle the problem. It is the only way that guarantee it won’t break again.

I wasn’t super clear in my previous answer but the problem is likely related to the way the %GAMEDIR% variable is resolved in FGenericPlatformMisc::ProjectDir . That method uses different techniques to resolve the project folder path as there are multiple scenarios to support:

  • Editor:
    • Native project: The project shares the same root as the Engine folder. Paths can be relative between the project and the engine. This is likely the case when you pass the project name only as argument.
    • Foreign project: The project doesn’t share the Engine folder root and can be on a different drive than the engine. Absolute paths will be used in this case.
  • Runtime: When the project is packaged, relative paths can be used since the packaging scripts aggregates everything under a common root.

I’m not exactly sure what is happening on your end as I only got relative paths in the Paths array in NormalizePackageNames. I tried running the commandlet with the project name and the uproject path. In both cases, the ProjectDir string is initialized through the FPaths::IsProjectFilePathSet path (line13354) in FGenericPlatformMisc::ProjectDir

[Attachment Removed]