Download

After migration UE4.22 -> 4.24 PSO Cache loading/running is not working

Hi guys!

We are having some difficulties with PSO caching in UE4.24.1.

We had everything fine about PSO caching with our game in UE 4.22 but after migration to 4.24 we can’t manage why the PSO cache is not running in the build.

We have stablepc.csv file already generated and it is already succesfully packaged to the project as we can see it loaded in the log while packaging, but probably we are missing some special command or config to get it loaded in the game. We followed the steps from the documenatation:
https://docs.unrealengine.com/en-US/…ing/index.html

DefaultDeviceProfiles.ini:
+CVars=r.ShaderPipelineCache.Enabled=1
+CVars=r.ShaderPipelineCache.StartupMode=0

We got three very simple BP functions in C++ and they are called in this order

In the BPGameInstance on EventInit:
FShaderPipelineCache::PauseBatching();

In the LoadingMap scene:
FShaderPipelineCache::SetBatchMode(FShaderPipelineCache::BatchMode::Fast);
FShaderPipelineCache::ResumeBatching();

pso2.png

In the same LoadingMap scene we check and wait in loop if the shaders are still compiling with:
FShaderPipelineCache::NumPrecompilesRemaining()

pso3.png

It looks to me that the problem looks like that PSO caching in 4.24 should be started manually with something different than FShaderPipelineCache::ResumeBatching() command but I can’t figure out what should that be.

Thanks!

Hello,

There is a new step introduced in 4.23 that has yet to be updated in the docs. (A docs ticket to update this info has been created)

From 4.23 release notes: “New: Generating stable shader info while cooking is now configurable. This data only needs to be generated if one or more platforms in your project use PSO caching. To do this, you will need to explicitly enable outputting shader stable keys by adding “NeedsShaderStableKeys=true” under the section “[DevOptions.Shaders]” in the engine .ini file for PSO platforms (for example, “AndroidEngine.ini” and “iOSEngine.ini”). This saves disk space and significant time (around 90 seconds in Epic’s use cases) on iterative cooks for large projects.”

Hi Alex,

thank you for your respond. However this is not the problem we are facing. We already have the pipeline caches file *.scl.csv generated and together with *.rec.upipelinecache files we already builded the file *.stablepc.csv which is needed for building the project with PSO cache in it.

The problem is that the PSO cache is loaded to the project while packaging (you can see the log) but when we run the build on the device the PSOs shaders wont compile. We are sure of that because of hitches in the game and also because the command ShaderPipelineCache::NumPrecompilesRemaining() is returning zero all the time.

It’s very confusing because we had everything perfect about PSO in UE 4.22.3 but when we switched to UE 4.24 it stopped working.

We’ve also tried on superempty project just only with several materials and meshes in it, just to have some shaders to be compiled but the result was the same. So I guess there should be some bug / changes about PSO in UE 4.24

I should mention first that we already had the configuration “NeedsShaderStableKeys=true” enabled for getting *.scl.csv.

This is our actual AndroidEngine.ini, but we tried a lot of configurations already. Still with no luck.

Config/Android/AndroidEngine.ini

*[SystemSettings]
r.ShaderPipelineCache.Enabled=1
r.ShaderPipelineCache.BatchSize=10
r.ShaderPipelineCache.BackgroundBatchSize=1
r.ShaderPipelineCache.BackgroundBatchTime=1
r.ShaderPipelineCache.BatchTime=10
r.ProgramBinaryCache.RestartAndroidAfterPrecompile=0

[DevOptions.Shaders]
NeedsShaderStableKeys=true*

I wish we could get this solved since its blocker for our project. Thank you!

Hi @jakub.resif

Did you find out a solution for this issue? I am currently in the same situation.

Thanks,

Matt

Hey Matt,

Fortunately we’ve found the solution. There is a bug in current version of UE4 with preoptimize mask which is causing this problem. So you need to disable the preoptimize mask code in “Engine/Source/Runtime/RenderCore/Private/ShaderPipelineCache.cpp” and compile the engine on your own. I’m attaching the edited file so you can just replace it in the folder and then compile the engine.

Don’t know if you have the environment for building engine already installed but just in case here are some links:

Githib account with epic account: Unreal Engine 4 on GitHub - Unreal Engine
UE4 github: https://github.com/EpicGames/UnrealEngine
Steps for installing : https://github.com/EpicGames/UnrealE…ease/README.md

Also some information I’ve found useful:
https://forums.unrealengine.com/deve…ease-answer-me

If you are on windows 7 as I am you will have to also download dot net 4.6.2, Windows 10 SDK, and if you are running on Visual studio 2019 I think you may have to go back to Visual studio 2017

Anyway we’ve been really lucky that someone internal from Epic did respond to us on github which we tried in desperation otherwise we’ll be still clueless. I wish there is a better support from Epic and guys around it for this cases since we always end up with no respond from every source we tried.

Wish you good luck,
Jakub

Hey @jakub.resif

Thanks for all the info! You’re a lifesaver. I’ll give it a try now.

Cheers,

Matt

EDIT:

I’m back to report our findings, and it does now seem to be precompiling the shaders correctly. (We’re basically do it the same way you mention in your opening post)

Did you try default pipeline for PSO in 4.25 w/o changes in source of engine? Is it works now?

As I know the Epic is planning to redesign the whole PSO thing in future but that means 4.25 is without any change.

I read your comment in github’s commit. Btw thanks.
And in latest commits I saw they work around upgrading pso for raytrace and virtual textures stuff.
I no have idea why ue doesn’t eat preprocessed stablepc.csv on packaging stage. As result - compile shaders in game always work on-demand with plenty hitches/freezes and nothing help to drive it.

Commandlets works only for generating upipeline and csv …
And simple plugin, like a BP wrapper for PSO, for experimenting - same result. Stablepc.csv looks like untouched on packaging. Always return 0 when try to unpause or switch modes.

Hi guys,
This CL will help you.
https://github.com/EpicGames/UnrealE…4e6eecd7a18366