Unable to modify Nav Data during PostBuild from custom UWorldPartitionNavigationDataBuilder

Hi!

At our project we are testing commandlets for nav data generation and manipulation.

We have a World Partition level and we use the commandlet to build the nav

"/Game/Developers/BrunoPistoni/Levels/WPTest_NavGen.umap" -run=WorldPartitionBuilderCommandlet -AllowCommandletRendering -builder=HexWorldPartitionNavigationDataBuilder -SCCProvider=None

In this case, HexWorldPartitionNavigationDataBuilder is our custom navigation data builder.

In it, we override PostRun with the idea of post-process the generated nav data to clean it out.

The way we do it is the following

// load the world (temp hack, propper cell-by-cell logic will be used)
const FBox BoxEntireWorld = FBox(FVector(-HALF_WORLD_MAX, -HALF_WORLD_MAX, -HALF_WORLD_MAX), FVector(HALF_WORLD_MAX, HALF_WORLD_MAX, HALF_WORLD_MAX));
TUniquePtr<FLoaderAdapterShape> LoaderAdapterShape = MakeUnique<FLoaderAdapterShape>(World, BoxEntireWorld, TEXT("Loaded Region"));
LoaderAdapterShape->Load();

// load every FHexNavFloodTask to start the nav flooding process
for (TActorIterator<AHexNavFloodReference> It(World); It; ++It)
{
	UPackage* Pkg = It->GetPackage();
	if (Pkg)
	{
		LoadPackage(nullptr, *SourceControlHelpers::PackageFilename(Pkg), LOAD_None);
	}
}

// look for every NavigationDataChunk in the world
TArray<FString> PackageFiles;
for (TActorIterator<ANavigationDataChunkActor> It(World); It; ++It)
{
	UPackage* Pkg = It->GetPackage();
	if (Pkg)
	{
		PackageFiles.AddUnique(SourceControlHelpers::PackageFilename(Pkg));
	}
}


// load each NavDataChunkActor
for (const FString& PackageFile : PackageFiles)
{
	FString LongPackageName;
	if (!FPackageName::TryConvertFilenameToLongPackageName(PackageFile, LongPackageName))
	{
		continue;
	}

	UPackage* LoadedPackage = LoadPackage(nullptr, *LongPackageName, LOAD_None);
	if (!LoadedPackage)
	{
		continue;
	}

	// Find the ANavigationDataChunkActor inside the package (if present)
	for (TObjectIterator<ANavigationDataChunkActor> ChunkIt; ChunkIt; ++ChunkIt)
	{
		if (ChunkIt->GetPackage() == LoadedPackage)
		{
			ANavigationDataChunkActor* ChunkActor = *ChunkIt;
			AHexRecastNavMesh* Recast = FindRecastNavMeshForChunk(ChunkActor); // we only use one nav data, so Recast is always the same object
			 
			if (Recast)
			{
				TObjectPtr<UHexNavFlooder> navFlooder = NewObject<UHexNavFlooder>(GetTransientPackage(), TEXT("navFlooder"));
				if (::IsValid(navFlooder))
				{
					//[do all our cleaning nav process, no need to show this code] navFlooder->FloodNav();

                                        // update the actor navData (is this needed?)
					FBox tileBounds(EForceInit::ForceInit);
					ChunkActor->CollectNavData(ChunkActor->GetBounds(), tileBounds);

					// Mark the package dirty so saving will persist changes
					LoadedPackage->SetDirtyFlag(true);
					Recast->GetPackage()->SetDirtyFlag(true);
				}
			}
		}
	}

	// find the packages we need to save and save them
	for (UPackage* Pkg : PackagesToSave)
	{
		FString PackageFilename;   
		if (FPackageName::TryConvertLongPackageNameToFilename(Pkg->GetName(), PackageFilename, Pkg->ContainsMap() ? FPackageName::GetMapPackageExtension() : FPackageName::GetAssetPackageExtension()))
		{
			SavePackageHelper(Pkg, PackageFilename);
		}
	}

	UPackage::WaitForAsyncFileWrites();

	// Now call base PostRun to handle final submission/notifications (or call it earlier if desired)
	return Super::PostRun(World, PackageHelper, bInRunSuccess);
}

But it doesn’t work. Our nav flood tool works in editor but not from within the commandlet. It feels like the changes to the nav are not being saved. We tried calling “ChunckActor->Destroy()” within the loop just to see what happens (expecting no nav data upon loading the level in editor), but no changes were observed upong opening the level in editor. The nav was just there, and we don’t allow for nav auto-generation. We checked the timestamp of the files in Windows to make sure the files are indeed changed. We think we are missing something key in the way we load/save the nav data, but not sure where and what.

Can you help us out figuring out what we need?

Any help or suggestion is very much appreciated

Thanks a lot!

[Attachment Removed]

Hi [mention removed]​,

Sorry for the delay, I have just been assigned the case. I’ll start looking into the case, if you have found any new information, please let us know :slightly_smiling_face:

Best,

Joan

[Attachment Removed]

Hi [mention removed]​,

I encountered some issues with the engine while testing this. However, I was able to successfully modify the nav data using a custom WorldPartitionNavigationDataBuilder.

In my case, I called the functionality directly from the RunInternal function instead of PostRun. It is possible that by the time the engine reaches PostRun, the NavMesh packages have already been processed and saved internally, so any later modifications do not take effect. I would need to confirm this to be completely sure.

By overriding RunInternal and then traversing the world’s ANavigationDataChunkActor instances using TActorIterator, I was able to modify the data and see the changes persist when launching the editor normally. I’m not sure if you have tested it using this approach, but at least in my case it worked as expected:

for (TActorIterator<ANavigationDataChunkActor> It(World); It; ++It)

I also tested this in a more recent development version of the engine. I will additionally test it in a clean 5.7 version to confirm the behavior. I noticed that in the latest commits there have been some changes and additional exposure in the Navigation Building System, which could potentially affect this behavior, but I still need to verify that.

I will update you shortly. As mentioned in my previous message, if you have gathered any new information in the meantime, please let me know.

Best regards,

Joan

[Attachment Removed]

Hi!

Sorry for the super-late reply, I got side-tracked with something else.

I tried in PreInit and I get the same result: nothing. I suspect I’m not manipulating the nav data correctly.

Can you show me what you are doing to manipulate the nav data (doesn’t matter if it’s what I want to do or not) so I can repro on my side and see if I get the change? I would imagine you are deleting every other nav poly or something similar. If I do the same and it works for me too, it will mean my actual logic for manipulation of the nav is wrong. If it doesn’t, it might mean we have some change locally that might be affecting our code.

Also, if you used a custom data builder, does that mean that you also had to change the source code to add the Export macro to the class UWorldPartitionNavigationDataBuilder? Because I had to (might be different in 5.7?)

Thank you!

[Attachment Removed]

Hi [mention removed]​,

No problem at all. I had to recreate the commandlets, and for now I’ve been testing by adding tags to the actors and renaming the label of all NavigationDataChunkActors.

One of the first commandlets I wrote removed some NavigationDataChunkActors and then rebuilt the navmesh. I remember running it, reopening the project afterwards, and seeing the navmesh changes reflected in the level. I’ll try to reproduce the nav data modifications again and update the thread once I have more results.

It would also be helpful if you could try the tag addition or label renaming on your side and check whether your NavigationDataChunkActors serialize those changes correctly.

I also had to add the export macro to UWorldPartitionNavigationDataBuilder, since without it Unreal does not allow the class to be used. For reference, it looks like this class is exported in newer engine versions, so developers will not need to add the export themselves going forward.

Below is the code I used to test renaming the actor labels:

.h

#pragma once
 
#include "CoreMinimal.h"
#include "WorldPartition/WorldPartitionNavigationDataBuilder.h"
 
#include "WorldPartition/WorldPartitionBuilder.h"
 
 
#include "HexWorldPartitionNavigationDataBuilder.generated.h"
 
UCLASS()
class EM_SC_API UHexWorldPartitionNavigationDataBuilder : public UWorldPartitionNavigationDataBuilder
{
	GENERATED_BODY()
 
public:
	virtual bool RequiresCommandletRendering() const override { return false; }
	virtual ELoadingMode GetLoadingMode() const override;
 
	virtual bool RunInternal(UWorld* World, const FCellInfo& InCellInfo, FPackageSourceControlHelper& PackageHelper) override;
	virtual bool PostRun(UWorld* World, FPackageSourceControlHelper& PackageHelper, const bool bInRunSuccess) override;
 
private:
 
private:
	bool SavePackageToDisk(UPackage* Package);
 
	// Collected in RunInternal, processed in PostRun
	TArray<FString> NavChunkPackages;
};

.cpp

#include "HexWorldPartitionNavigationDataBuilder.h"
 
#include "WorldPartition/NavigationData/NavigationDataChunkActor.h"
 
#include "Engine/World.h"
#include "Misc/PackageName.h"
#include "UObject/Package.h"
#include "UObject/SavePackage.h"
 
#include "WorldPartition/WorldPartition.h"
#include "WorldPartition/WorldPartitionHelpers.h"
 
static ANavigationDataChunkActor* FindChunkActorInPackage(UPackage* Package)
{
    if (!Package) return nullptr;
 
    ANavigationDataChunkActor* Result = nullptr;
 
    ForEachObjectWithPackage(Package, [&](UObject* Obj) -> bool
        {
            if (auto* A = Cast<ANavigationDataChunkActor>(Obj))
            {
                Result = A;
                return false; 
            }
            return true; 
        },  true);
 
    return Result;
}
 
UWorldPartitionBuilder::ELoadingMode UHexWorldPartitionNavigationDataBuilder::GetLoadingMode() const
{
    return ELoadingMode::Custom;
}
 
bool UHexWorldPartitionNavigationDataBuilder::RunInternal(UWorld* World, const FCellInfo& InCellInfo, FPackageSourceControlHelper& PackageHelper)
{
    NavChunkPackages.Reset();
 
    if (!World)
    {
        UE_LOG(LogTemp, Error, TEXT("[HexNavBuilder] World is null"));
        return false;
    }
 
    UWorldPartition* WP = World->GetWorldPartition();
    if (!WP)
    {
        UE_LOG(LogTemp, Error, TEXT("[HexNavBuilder] World has no WorldPartition"));
        return false;
    }
 
    UE_LOG(LogTemp, Warning, TEXT("[HexNavBuilder] RunInternal World=%s WorldType=%d"),
        *World->GetName(), (int32)World->WorldType);
 
    int32 DescCount = 0;
 
    FWorldPartitionHelpers::ForEachActorDescInstance<ANavigationDataChunkActor>(
        WP,
        [&](const FWorldPartitionActorDescInstance* Desc)
        {
            if (!Desc) return true;
 
            ++DescCount;
            const FString Pkg = Desc->GetActorPackage().ToString();
            NavChunkPackages.AddUnique(Pkg);
 
            UE_LOG(LogTemp, Warning, TEXT("[HexNavBuilder] NavChunkDesc[%d] Name=%s Pkg=%s"),
                DescCount,
                *Desc->GetActorName().ToString(),
                *Pkg);
 
            return true;
        });
 
    UE_LOG(LogTemp, Warning, TEXT("[HexNavBuilder] Collected %d unique nav chunk packages"), NavChunkPackages.Num());
    return true;
}
 
bool UHexWorldPartitionNavigationDataBuilder::PostRun(UWorld* World, FPackageSourceControlHelper& PackageHelper, const bool bInRunSuccess)
{
    const bool bBaseOk = Super::PostRun(World, PackageHelper, bInRunSuccess);
 
    if (!World)
    {
        UE_LOG(LogTemp, Error, TEXT("[HexNavBuilder] PostRun World is null"));
        return false;
    }
 
    UE_LOG(LogTemp, Warning, TEXT("[HexNavBuilder] PostRun processing %d chunk packages"), NavChunkPackages.Num());
 
    bool bAllOk = true;
    int i = 0; 
 
    for (const FString& PkgName : NavChunkPackages)
    {
        UE_LOG(LogTemp, Warning, TEXT("[HexNavBuilder] Loading %s"), *PkgName);
 
        UPackage* Pkg = LoadPackage(nullptr, *PkgName, LOAD_None);
        if (!Pkg)
        {
            UE_LOG(LogTemp, Warning, TEXT("[HexNavBuilder] FAILED to load %s"), *PkgName);
            bAllOk = false;
            continue;
        }
 
        ANavigationDataChunkActor* Chunk = FindChunkActorInPackage(Pkg);
        if (!Chunk)
        {
            UE_LOG(LogTemp, Warning, TEXT("[HexNavBuilder] No ANavigationDataChunkActor found inside %s"), *PkgName);
            bAllOk = false;
            continue;
        }
 
        
		FString NewName = FString::Printf(TEXT("My_Actor_Label_%d"), i+1);
        Chunk->Modify();
        Chunk->SetActorLabel(NewName, false); 
        Pkg->MarkPackageDirty();
        bool bSaved = SavePackageToDisk(Pkg);
        i++;
        UE_LOG(LogTemp, Warning, TEXT("[HexNavBuilder] Save %s : %s"), *PkgName, bSaved ? TEXT("OK") : TEXT("FAILED"));
 
 
    }
 
    UPackage::WaitForAsyncFileWrites();
 
    return bBaseOk && bAllOk;
    
}
 
bool UHexWorldPartitionNavigationDataBuilder::SavePackageToDisk(UPackage* Package)
{
    if (!Package) return false;
 
    FString Filename;
    const bool bOk = FPackageName::TryConvertLongPackageNameToFilename(
        Package->GetName(),
        Filename,
        Package->ContainsMap() ? FPackageName::GetMapPackageExtension()
        : FPackageName::GetAssetPackageExtension());
 
    if (!bOk)
    {
        UE_LOG(LogTemp, Error, TEXT("[HexNavBuilder] Could not resolve filename for %s"), *Package->GetName());
        return false;
    }
 
    FSavePackageArgs MyArgs;
    MyArgs.TopLevelFlags = RF_Public | RF_Standalone;
    MyArgs.SaveFlags = SAVE_NoError;
 
    const bool bSaved = UPackage::SavePackage(Package, nullptr, *Filename, MyArgs);
    return bSaved;
}

I tested this project with ue5-main branch, so it could be that in new versions of the engine your issue was fixed. I’ll compile now the 5.6 source version and test again and see if it still works. Will update shortly.

Best Regards,

Joan

[Attachment Removed]