This question was created in reference to: [Modifying levels during [Content removed] [Modifying WorldPartition levels during cook using [Content removed]
I am trying to gather some data about actors from a level (enumerate all actors with a specific component, read some properties from it, save soft link to actor and some of the properties in some database). Trivial for simple levels, however I am having issues with World Partition levels (that use external actors and data layers).
It seems that this sort of preprocessed data is usually saved to an actor in level (navmesh, persistent smart object collection, etc…), so I tried using the same method - single actor in level, that has some data saved on in. I have made this actor not spatially loaded, so that it would be present in the level at all times.
During level cooking, the base level with all it’s actors is cooked and saved before ANY streamable actors are even loaded. Meaning that at the time my actor is processed, I cannot gather any data, because nothing is loaded.
I have tried moving this data gathering to world subsystem PreSave - all of the actors are loaded at that point, however the base level is already saved, so I can generate the data now, but I cannot export it anywhere.
Can I resave the package? Or delay the saving of the base package after all actors have been loaded? Or force all WP cells to load before the main package is saved? Or save the data somewhere else than a level actor?
(This absolutely needs to be automated, so running a manual editor preprocess step is out of the question)
You can use `IAssetRegistry::GetAssetsByTagValues` or `FARFilter::TagsAndValues` to search the Asset Registry for the assets that match the desired tags.
Another option could be to create a UWorldPartitionRuntimeCellTransformer which allow manipulating the cells in a World Partition level when running PIE or cooking.
Can you clarify the use case? What type of data are you trying to gather and for which purpose? It would be easier to assist if we understand what you are trying to build.
I am looking for all actors with one specific component. This component has a list of {softObject reference to actor + some primitive parameters}. I want to save all of this information to a"database", so that I can look it up for any actor that happens to be unloaded by world partition (thats why I can’t save the data into the world partition cell itself, i need it most when the cell is unloaded).
The ultimate purpose is some approximate simulation of the parts of the world that is are far from the player.
I looked into UWorldPartitionRuntimeCellTransformer, but it seemed that they are called when the cell is loaded, which doesn’t really help me - at that point it is too late to save any data (in my current solution).
The asset registry is more interesting. Are you suggesting that during some early cook step (probably while cooking the base level), I just load any actors that I need through the asset registry? (There will probably be some issues with needing to tag all my actors, as I cannot filter on components???, but that is hopefully solvable)
Hello, jumping in here after doing some testing. I was one of the people who mentioned WorldPartitionRuntimeCellTransformer to your team in the past. One thing I want to emphasize is that the WorldPartitionRuntimeCellTransformer are ran at cook-time so is a valid route for preprocessing worlds for cooked builds.
However, I’d like to update that recommendation since I understand a bit better what you’re trying to do and it would be convenient to be able to calculate that database at editor-time too without cooking a build. I think a World Partition Builder Commandlet is what you need for preprocessing the world. There are many examples in the engine where we use this to calculate data offline, including a commandlet that loads world partition cells one-by-one to generate navigation data for that cell.
UWorldPartitionNavigationDataBuilder and UWorldPartitionResaveActorsBuilder should be good reference to you. UWorldPartitionResaveActorsBuilder in particular demonstrates how to resave all actors with a commandlet and I think is a good template for you to do actor preprocessing instead (doesn’t have to resave or modify the actors then, but rather to export to your database). My suggestion for you is to create a custom UWorldPartitionBuilder class. You can then from commandline (outside of UnrealEditor) let that builder class run on an open world map and it gets to parse all actors in all cells. Here is an example:
`#pragma once
include “CoreMinimal.h” include “WorldPartition/WorldPartitionBuilder.h” include “MyWorldPartitionDatabaseBuilder.generated.h”
/**
*
*/
UCLASS()
class THIRDPERSON55R_API UMyWorldPartitionDatabaseBuilder : public UWorldPartitionBuilder
{
GENERATED_BODY()
UWorldPartitionBuilder::ELoadingMode UMyWorldPartitionDatabaseBuilder::GetLoadingMode() const
{
// Load cells one by one, calls RunInternal with world partially loaded multiple times
return ELoadingMode::IterativeCells2D;
}
bool UMyWorldPartitionDatabaseBuilder::RunInternal(UWorld* World, const FCellInfo& InCellInfo, FPackageSourceControlHelper& PackageHelper)
{
UE_LOG(LogTemp, Warning, TEXT(“RunInternal called on world: %s”), *World->GetPathName());
for (TActorIterator ItActor(World); ItActor; ++ItActor)
{
UE_LOG(LogTemp, Warning, TEXT(“STUB: Parse actor here - %s”), *ItActor->GetPathName());
}
return true;
}`
And running the commandlet:
"Z:/My/Path/Engine/Binaries/Win64/UnrealEditor.exe" "%cd%/ThirdPerson55R.uproject" "/Game/ThirdPerson/Maps/MAP_MyOpenWorld" -run=WorldPartitionBuilderCommandlet -SCCProvider=None -builder=MyWorldPartitionDatabaseBuilderThat should help you preprocess actors at least to extract any info you want and store in a custom database. You can detect here which actors have that component. Depending on what you return as GetLoadingMode(), you can have all actors loaded (memory heavy of course), or just actors in one cell at a time. If actors have soft object references to each other, then EntireWorld may be more appropriate if you want to have the other actor loaded too, but it might not be feasible depending on the size of your map so there is a pro and con. Please let us know if this brings you closer to what you need.
One thing to be aware of is that when running a WorldPartitionBuilder and getting the actor’s paths, all actors are loaded into the PersistentLevel. They may have a path like this:
/Game/ThirdPerson/Maps/MAP_MyOpenWorld.MAP_MyOpenWorld:PersistentLevel.StaticMeshActor_UAID_E04F43E60CEA876402_1193738857At runtime in PIE and in cooked builds they will have different paths. In PIE there will be a UEDPIE prefix, in cooked builds those actors will be loaded as part of a sublevel (a world partition cell) so not in the Persistent Level. Soft object reference properties of actors placed in the level will be fixed up so the paths work in PIE and in cooked builds, but if you copy actor paths into your custom database you will need to fix them up at runtime. I’ve asked a colleague to pitch in with what they know about that.
“The asset registry is more interesting.”
Regarding this idea, I’m not too familiar myself with using asset registry to find actors placed in the level. I believe that approach is more suitable to find blueprint assets and data assets, rather than one-file-per-actor actor assets.
As my colleague Zhi mentioned, the actor’s paths will change at runtime. This happens during the runtime streaming generation that is done at PIE/cook time, when all of the streaming cells/sublevels get generated, and the actors are placed into their appropriate cells. During this, any soft object references to the actors are also automatically remapped.
If you need to get the runtime path from an editor path at runtime, a helper function can be used: FWorldPartitionHelpers::ConvertEditorPathToRuntimePath.
Thank you for your answer, but I am afraid the commandlet has some huge disadvangtages - mainly that it is a separate process that needs to run over all of the data, and save the data into some “database”.
I need the system to work automatically on each game cook/PIE start (in PIE, we probably can run the process for some cells and ignore the far away ones). The idea that a manual action would be needed before every PIE start is just unworkable, our scripters would have my head for this.
I am also trying to save the data in some actor and not an actual independent asset. In our previous project (not unreal) we ended up with 5 different preprocessed data sets - I really want to avoid needing to create and link 5 very specific assets for each level you create.
One solution I found was to create an Actor (that is not Spatially loaded) and in it’s PreSave iterate over all ActorDescContainerInstances, load all the actors, and gather the data. I am afraid that for larger level cooks, loading actors out of order like this is going to slow down the cook quite a bit.
For that reason, I am trying to move the logic into WorldPartitionRuntimeCellTransformer - however I am running back into the original issue - World cells are cooked after the level is cooked and saved and I cannot insert any data into the base level at that point.
Is there a way to force the base level to resave? Or even better, Is there a place where I could put the data, so that it would be available from when the cell is unstreamed? It seems to be possible to force-load one specific actor from a world cell, if I have it’s SoftObjectPath - so I could save all the data for one cell into a specific actor and then force-load it, leaving the rest of the cell unloaded. Or somehow store UserAssetData on the UWorldPartitionRuntimeLevelStreamingCell?
I apologize for the delay. We plan to investigate having cell transformers work on the persistent level; however, we don’t have a timeline to share if/when this might be expected. My colleague who will be looking into it hasn’t had a chance this week due to other tasks, but if there are any updates within a reasonable amount of time from now, then I’ll let you know. For now, it seems best to roll your own solution, such as the one you found.
I have in the meantime come up with following solution:
I have a Preprocess actor, which in it’s PreSave gathers all data from it’s level, and saves the data onto itself.
For the persistent level, this actor is manually pre-placed. For WP cells, the cell transformer creates an actor with a fixed name.
When starting the game, I iterate over all existing actors to find the pre-placed ones, and then force load the transformer-placed ones (I have the package name for each WP cell, and the name of the actor - I use that to construct the path and request loading of that actor). Then I gather the data from all these actors to one centralized location and leave the actors alone (I never add them to the word, so presumably they will just be garbage collected later).