Solution for landscape popping when using PSO deferred scene proxy creation strategy

Development Rendering
, ,
1

Hello!

We have been seeing the same landscape related PSO hitches discussed in this thread: [Missing Landscape [Content removed]

I had actually implemented the same fix in 5.7 before finding this thread and realizing it would be ‘fixed’ in 5.8. We haven’t moved our project to 5.8 yet, but AFAIK there is not a solution to the landscape pop-in / holes that the deferral strategy causes in 5.8. The pop-in is very prominent and means this change on its own doesn’t resolve the issue for us.

I have been investigating potential solutions here, the most promising of which would be delaying the hiding of the HLOD for the cell containing the landscape components that trigger the PSO precache. I have a prototype which works well but we have risk concerns due to the complexity of the surrounding systems, and haven’t determined the ‘correct’ implementation yet.

The current implementation works by having the landscape components (which could be genericized to UPrimitiveComponent) inform the UWorldPartitionHLODRuntimeSubsystem that the streaming cell containing them has a pending scene proxy creation (which is deferred due to the PSO precache). When the UWorldPartitionHLODRuntimeSubsystem is told that this cell is shown (and thus it should hide the HLOD), if there is pending scene proxy creation then it defers the hide until all the components in that cell have created the scene proxy.

This works but we’re not sure what the best injection point is for the component to tell the HLOD subsystem there is a pending scene proxy creation (and subsequently clear that flag). CreateSceneProxy can be called from multiple threads and takes place during the render update, so there’s potential for race conditions. We have also seen that UWorldPartitionHLODRuntimeSubsystem has a warmup system that (somewhat counter-intuitively) also has a warm-down delay for when the HLOD would be hidden (UWorldPartitionHLODRuntimeSubsystem::CanMakeVisible), so we’re thinking this could potentially check the PSO state of the cell’s components so CanMakeInvisible returns false until the precache has completed. But I haven’t looked into this route in detail yet.

I’m mainly asking what you think the best approach here is. Assuming there’s not already some solution in 5.8 that I’ve missed. What advice do you have on solving this problem? Thanks!

2

Hi Sam,

Thanks for reaching out. We are still discussing how to move forward, since your comment has prompted us to explore extending the HLOD system to better accommodate PSO precaching. The main expert on the precaching system is still out of the office for two weeks, so it will take some time until we have a definite response on how we want to proceed. However, your approach seems fine for now, so long as you are currently not running into any issues. Please let me know if you have any questions or feedback, and I will try to update you as soon as I have any news.

3

Thank you for the update Tim!

I think we have converged on a solution that seems clean and safe and works for us. Note that this is on 5.7, in 5.8 we see there is a new FPSOPrecacheComponent system that will require a few updates but the principles should remain.

UWorldPartitionHLODRuntimeSubsystem gets a pair of methods that increment/decrement a per-cell counter that represents how many PSO precache tasks we’re currently waiting on in that cell. Inside OnCellShown, if there’s a pending PSO precache then we don’t immediately hide the HLODs. When NotifyComponentProxyCreated for the final PSO precache task is called, it hides the HLODs.

HLODRuntimeSubsystem.h

ENGINE_API void NotifyComponentProxyPending(const UWorldPartitionRuntimeCell* Cell);
ENGINE_API void NotifyComponentProxyCreated(const UWorldPartitionRuntimeCell* Cell);

struct FCellData
	{
		bool bIsCellVisible = false;
		TArray<IWorldPartitionHLODObject*> LoadedHLODs;	// HLOD representation of the cell itself
		int32 ComponentsPendingPSOPrecacheCount = 0;
	};

HLODRuntimeSubsystem.cpp

void UWorldPartitionHLODRuntimeSubsystem::NotifyComponentProxyPending(const UWorldPartitionRuntimeCell* Cell)
{
#if UE_WITH_PSO_PRECACHING
	check(IsInGameThread());
 
	if (CVarShowHLODsDuringPSOPrecache.GetValueOnAnyThread() == 0)	return;
 
	if (!Cell) return;
 
	if (FCellData* CellData = GetCellData(Cell))
	{
		CellData->ComponentsPendingPSOPrecacheCount++;
	}
#endif
}
 
void UWorldPartitionHLODRuntimeSubsystem::NotifyComponentProxyCreated(const UWorldPartitionRuntimeCell* Cell)
{
#if UE_WITH_PSO_PRECACHING
	check(IsInGameThread());
 
	if (CVarShowHLODsDuringPSOPrecache.GetValueOnAnyThread() == 0)	return;
 
	if (!Cell) return;
 
	if (FCellData* CellData = GetCellData(Cell))
	{
		// Because we reset the counter in OnCellHidden, if cell is hidden during an in-flight PSO precache, it can be decremented when
		// already 0, so clamp it.
		CellData->ComponentsPendingPSOPrecacheCount = FMath::Max(0, CellData->ComponentsPendingPSOPrecacheCount - 1);
		if (CellData->ComponentsPendingPSOPrecacheCount == 0 && CellData->bIsCellVisible)
		{
			for (IWorldPartitionHLODObject* HLODObject : CellData->LoadedHLODs)
			{
				HLODObject->SetVisibility(false);
			}
		}
	}
#endif
}

Then, in OnCellShown:

#if UE_WITH_PSO_PRECACHING
			if (CVarShowHLODsDuringPSOPrecache.GetValueOnAnyThread() == 1)
			{
				check(IsInGameThread());
				if (CellData->ComponentsPendingPSOPrecacheCount > 0)
				{
					return;
				}
			}
#endif

and OnCellHidden:

#if UE_WITH_PSO_PRECACHING
		if (CVarShowHLODsDuringPSOPrecache.GetValueOnAnyThread() == 1)
		{
			check(IsInGameThread());
			// This resolves an edge-case if a cell is destroyed while a PSO precache is in-flight, NotifyComponentProxyCreated won't be called
			// so the counter will accumulate a stale increment that will prevent HLODs from hiding. So reset the counter when a cell is hidden.
			CellData->ComponentsPendingPSOPrecacheCount = 0;
		}
#endif

The notify events are called in UPrimitiveComponent::RequestRecreateRenderStateWhenPSOPrecacheFinished:

			const UWorldPartitionRuntimeCell* Cell = nullptr;
			if (UWorld* World = GetWorld())
			{
				if (const AActor* Owner = GetOwner())
				{
					if (const ULevel* Level = Owner->GetLevel())
					{
						Cell = Cast<UWorldPartitionRuntimeCell>(Level->GetWorldPartitionRuntimeCell());
						if (Cell)
						{
							if (UWorldPartitionHLODRuntimeSubsystem* Subsystem = World->GetSubsystem<UWorldPartitionHLODRuntimeSubsystem>())
							{
								// Only notify if on game thread, to avoid having to handle race conditions on the HLOD subsystem counter. In practice I only saw NiagaraComponent
								// call this on non-GT. If not in GT, Cell = null so the paired NotifyComponentProxyCreated won't be called and balance is maintained.
								if (IsInGameThread())
								{
									Subsystem->NotifyComponentProxyPending(Cell);
								}
								else
								{
									Cell = nullptr;
								}
							}
						}
					}
				}
			}
			TGraphTask<FPSOPrecacheFinishedTask>::CreateTask(&PSOPrecacheCompileEvents).ConstructAndDispatchWhenReady(this, LatestPSOPrecacheJobSet, Cell);

And in FPSOPrecacheFinishedTask::DoTask:

if (const UWorldPartitionRuntimeCell* Cell = WeakRuntimeCell.Get())
		{
			if (UWorld* World = Cell->GetWorld())
			{
				if (UWorldPartitionHLODRuntimeSubsystem* Subsystem = World->GetSubsystem<UWorldPartitionHLODRuntimeSubsystem>())
				{
					Subsystem->NotifyComponentProxyCreated(Cell);
				}
			}
		}

The one edge-case is this doesn’t work for NiagaraComponents because it calls its RequestRecreateRenderStateWhenPSOPrecacheFinished from a task thread which could cause race conditions mutating the counter. So we just skip anything coming from a non-game thread. This could be multi-threaded with an atomic counter but there’s some extra complications with that since FCellData is stored in a TMap.

4

Hello again, we are still discussing this request internally, but we have not reached an agreement on some of the technical details of the changes. I have filed this as an official feature request with the rendering team so we can track this effort. Since you have a working solution for yourselves, you can keep using it until we release our own implementation. I hope that works for you, but let me know if you have any more questions.

5

Ok, good to hear! Then I will close out the ticket, and I can try to update you once we have something official in place

6

Thanks Tim! It’s been working well for us since submit so we’ll continue using it, no rush on an official fix.