PSO Precaching Misses for Landscape

Development Rendering
, ,
1

Hello,

We created a full PSO bundle, but still receive the following issues:

  1. PSOs are precached at runtime. This is in general ok, since pre-caching is enabled, but I am wondering if this should happen at all when having a bundled PSO cache
  2. We are also receiving Preaching Misses as shown below

PSO PRECACHING MISS:

Type: FullPSO

PSOPrecachingState: Too Late

Material: LandscapeMaterialInstanceConstant_12

VertexFactoryType: FLandscapeFixedGridVertexFactory

MDCStatsCategory: Unknown

MeshPassName: VirtualTexture

Shader Hashes:

VertexShader:    0336001A46EEF21A0883EB8C60E77CE79405FEFB

PixelShader:    7E9F155A7B13B3694C71FAF53084545740F373DC

It’s hard to understand where these misses are coming from, can you help us to find out?

Kind regards

Moritz Grunwald

2

Hello,

Thank you for reaching out.

I’ve been assigned this issue, and we will be looking into these PSO Precache misses for you.

3

Hello,

I was further investigating the issue and found some related UDN threads:

  • [Content removed]
  • [Content removed]

I am still checking some thing, but it looks like D3D12.PSO.KeepUsedPSOsInLowLevelCache=1 is fixing the issue. I will adjust this post when I have more info.

4

Hello,

This ticket discusses using both PSO Precaching and Bundled PSO Caches at the same time:

[Content removed]

This documentation discusses two CVars you can use to control how PSO Precaching and Bundled PSOs work together:

https://dev.epicgames.com/documentation/en\-us/unreal\-engine/manually\-creating\-bundled\-pso\-caches\-in\-unreal\-engine\#interactionsbetweenmanualpsocacheandrun\-timepsoprecaching

This documentation provides information and CVars to debug PSO Precache misses.

https://dev.epicgames.com/documentation/en\-us/unreal\-engine/pso\-precaching\-for\-unreal\-engine\#debug\-a\-pso\-precache\-miss

For your use-case do you have an initial loading implementation for precompiling PSOs?

Please let us know if this helps.

5

Hello,

thank you for the links to the documentation, but I was already aware of them. Our use case is that we have a full bundled PSO Cache that starts compiling when the game starts (r.ShaderPipelineCache.StartupMode=1) and also keeps our loading screen shown when the player starts a game. The idea is to cover everything with the bundle and use PSP Precaching as a fallback. The PSO misses mentioned above are appearing after the loading screen has finished and all PSOs should have been compiled

LogRHI: Warning: FShaderPipelineCache OurGameName completed 3983 tasks in 25.41s (169.80s wall time since intial open).

I was debugging some of the PSO misses and it looks like they are coming from PSOs being thrown away. I had to use the following CVars to prevent this:

D3D12.PSO.KeepUsedPSOsInLowLevelCache=1

D3D12.PSOPrecache.KeepLowLevel=1

r.PSOPrecache.KeepInMemoryUntilUsed=1

r.pso.evictiontime=0

But I also read some comments regarding these variables that they can lead to OOM because too much shaders are stored, so not sure if we will use them in the end. Anyways, using these CVars seems to fix the “Too Late” PSO misses, but we still see others:

PSO PRECACHING MISS:
	Type:					FullPSO
	PSOPrecachingState:		Missed
	Material:				Unknown
	VertexFactoryType:		None
	MDCStatsCategory:		Unknown
	MeshPassName:			SlateMaterialPSOCollector
	Shader Hashes:
		VertexShader:		54B72D0E8FC6D0BB3B09D6C4613B956335439160
		PixelShader:		CBE41A05F7BC7046A2271A1C4CCE27FEE930A939
 
	No material found so no extra information on miss

These are, as far as I can see, caused my not matching RenderTargetFlags (Looks like these PSOs have no flags in the bundle).

PSO PRECACHING MISS:
	Type:					ShadersOnly
	PSOPrecachingState:		Missed
	Material:				MI_OutfitFemale01_Lux
	VertexFactoryType:		TGPUSkinVertexFactoryUnlimited
	MDCStatsCategory:		SkeletalMeshComponentBudgeted
	MeshPassName:			BasePass
	Shader Hashes:
		VertexShader:		360D5B8170ED46ADC2F4C458E6CD86F73644EA6A
		PixelShader:		2B899FA8BA6C743ADE3EC81C87A11876A01931C8
 
	Missed Info:
		Precached with:		TGPUSkinVertexFactoryDefault (PSOPrecacheParamData: 4288000)
		Precached with:		TGPUSkinVertexFactoryDefault (PSOPrecacheParamData: 4288000)

And these are caused by a missmatching Vertex Factory (PSO in bundle has TGPUSkinVertexFactoryDefault and the one requested at runtime is using TGPUSkinVertexFactoryUnlimited).

6

ok, after diving deep into the code I noticed, what was already mentioned here [Content removed] The PSO Bundle and the PSO Precaching are using different caches (GGraphicsPipelineCache and GPrecacheGraphicsPipelineCache in PipelineStateCache.cpp). I always thought the same cache is used, but now I have new questions:

  1. Does this mean PSO PRECACHING MISSES cannot be tackled with a cached PSO Bundle?
  2. Does PSO Precaching will then always create tasks for compiling and ignore the PSO Bundle (looks like it during debugging and profiling)?
7

Hello,

“GGraphicsPipelineCache” and “GPrecacheGraphicsPipelineCache” serve different purposes (these are the same as the Compute versions):

  • “GPrecacheGraphicsPipelineCache” - stores the precache state for a shader. This is an enum value, determining if the shader’s state is Done, Missed, Active, etc. See “EPSOPrecacheResult” from PipelineStateCache.h.
  • “GGraphicsPipelineCache” - caches the PSOs actually being used by the game.

Keep in mind that “r.PSOPrecache.KeepInMemoryUntilUsed” only applies to NVIDIA hardware. For more information, please see “ShouldKeepPrecachedPSOsInMemory()” from PipelineStateCache.cpp.

The “GGraphicsPipelineCache” is separate from the D3D12 low-level cache, which is controlled by the two “D3D12.*” CVars you posted. This cache is used by “RHICreateGraphicsPipelineState(…)” and “RHICreateComputePipelineState(…)”. For more information, please see their definitions in D3D12State.cpp, and the usage of “FD3D12PipelineStateCache”.

These misses you presented are the result of the PSO collection not marking the correct PSO for Precaching. You might need to customize “ShouldPrecachePermutation(…)” for Global Shaders or “CollectPSOPrecacheData(…)” for Primitive Components to handle any permutations that are not currently handled.

Please let us know if this helps.

8

Hello,

thank you for the detailed explanation. This helps a lot!

Kind regards

Moritz Grunwald

9

Hello,

Thank you for the reply.

Can we close your case? You can always re-open it if you have additional questions about the same issue.

10

Hello,

yes, this case can be closed.