Announcement

Collapse
No announcement yet.

After migration UE4.22 -> 4.24 PSO Cache loading/running is not working

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    After migration UE4.22 -> 4.24 PSO Cache loading/running is not working

    Hi guys!

    We are having some difficulties with PSO caching in UE4.24.1.

    We had everything fine about PSO caching with our game in UE 4.22 but after migration to 4.24 we can't manage why the PSO cache is not running in the build.

    We have stablepc.csv file already generated and it is already succesfully packaged to the project as we can see it loaded in the log while packaging, but probably we are missing some special command or config to get it loaded in the game. We followed the steps from the documenatation:
    https://docs.unrealengine.com/en-US/...ing/index.html

    DefaultDeviceProfiles.ini:
    +CVars=r.ShaderPipelineCache.Enabled=1
    +CVars=r.ShaderPipelineCache.StartupMode=0

    We got three very simple BP functions in C++ and they are called in this order

    1.
    In the BPGameInstance on EventInit:
    FShaderPipelineCache::PauseBatching();

    Click image for larger version  Name:	pso1.png Views:	6 Size:	16.4 KB ID:	1707968

    2.
    In the LoadingMap scene:
    FShaderPipelineCache::SetBatchMode(FShaderPipelineCache::BatchMode::Fast);
    FShaderPipelineCache::ResumeBatching();

    Click image for larger version  Name:	pso2.png Views:	5 Size:	18.8 KB ID:	1707969

    3.
    In the same LoadingMap scene we check and wait in loop if the shaders are still compiling with:
    FShaderPipelineCache::NumPrecompilesRemaining()

    Click image for larger version  Name:	pso3.png Views:	5 Size:	24.6 KB ID:	1707970

    It looks to me that the problem looks like that PSO caching in 4.24 should be started manually with something different than FShaderPipelineCache::ResumeBatching() command but I can't figure out what should that be.

    Thanks!


    Last edited by jakub.resif; 01-13-2020, 11:10 AM.

    #2
    Hello,

    There is a new step introduced in 4.23 that has yet to be updated in the docs. (A docs ticket to update this info has been created)

    From 4.23 release notes: "New: Generating stable shader info while cooking is now configurable. This data only needs to be generated if one or more platforms in your project use PSO caching. To do this, you will need to explicitly enable outputting shader stable keys by adding "NeedsShaderStableKeys=true" under the section "[DevOptions.Shaders]" in the engine .ini file for PSO platforms (for example, "AndroidEngine.ini" and "iOSEngine.ini"). This saves disk space and significant time (around 90 seconds in Epic's use cases) on iterative cooks for large projects."

    Comment


      #3
      Hi Alex,

      thank you for your respond. However this is not the problem we are facing. We already have the pipeline caches file *.scl.csv generated and together with *.rec.upipelinecache files we already builded the file *.stablepc.csv which is needed for building the project with PSO cache in it.

      The problem is that the PSO cache is loaded to the project while packaging (you can see the log) but when we run the build on the device the PSOs shaders wont compile. We are sure of that because of hitches in the game and also because the command ShaderPipelineCache::NumPrecompilesRemaining() is returning zero all the time.

      Click image for larger version  Name:	20200121-22_57_17testpso--Unreal-Editor.png Views:	0 Size:	39.3 KB ID:	1711536

      It's very confusing because we had everything perfect about PSO in UE 4.22.3 but when we switched to UE 4.24 it stopped working.

      We've also tried on superempty project just only with several materials and meshes in it, just to have some shaders to be compiled but the result was the same. So I guess there should be some bug / changes about PSO in UE 4.24

      I should mention first that we already had the configuration "NeedsShaderStableKeys=true" enabled for getting *.scl.csv.

      This is our actual AndroidEngine.ini, but we tried a lot of configurations already. Still with no luck.

      Config/Android/AndroidEngine.ini

      [SystemSettings]
      r.ShaderPipelineCache.Enabled=1
      r.ShaderPipelineCache.BatchSize=10
      r.ShaderPipelineCache.BackgroundBatchSize=1
      r.ShaderPipelineCache.BackgroundBatchTime=1
      r.ShaderPipelineCache.BatchTime=10
      r.ProgramBinaryCache.RestartAndroidAfterPrecompile=0

      [DevOptions.Shaders]
      NeedsShaderStableKeys=true


      I wish we could get this solved since its blocker for our project. Thank you!
      Last edited by jakub.resif; 01-21-2020, 06:45 PM.

      Comment


        #4
        Hi jakub.resif

        Did you find out a solution for this issue? I am currently in the same situation.

        Thanks,

        Matt
        http://www.poolnationfx.com/
        http://www.poolnationvrgame.com/

        Comment


          #5
          Hey Matt,

          Fortunately we've found the solution. There is a bug in current version of UE4 with preoptimize mask which is causing this problem. So you need to disable the preoptimize mask code in "Engine/Source/Runtime/RenderCore/Private/ShaderPipelineCache.cpp" and compile the engine on your own. I'm attaching the edited file so you can just replace it in the folder and then compile the engine.

          Don't know if you have the environment for building engine already installed but just in case here are some links:

          Githib account with epic account: https://www.unrealengine.com/en-US/ue4-on-github
          UE4 github: https://github.com/EpicGames/UnrealEngine
          Steps for installing : https://github.com/EpicGames/UnrealE...ease/README.md

          Also some information I've found useful:
          https://forums.unrealengine.com/deve...ease-answer-me

          If you are on windows 7 as I am you will have to also download dot net 4.6.2, Windows 10 SDK, and if you are running on Visual studio 2019 I think you may have to go back to Visual studio 2017

          Anyway we've been really lucky that someone internal from Epic did respond to us on github which we tried in desperation otherwise we'll be still clueless. I wish there is a better support from Epic and guys around it for this cases since we always end up with no respond from every source we tried.

          Wish you good luck,
          Jakub
          Attached Files
          Last edited by jakub.resif; 02-29-2020, 09:32 AM.

          Comment


            #6
            Hey jakub.resif

            Thanks for all the info! You're a lifesaver. I'll give it a try now.

            Cheers,

            Matt

            EDIT:

            I'm back to report our findings, and it does now seem to be precompiling the shaders correctly. (We're basically do it the same way you mention in your opening post)
            Last edited by Milly01; 03-02-2020, 02:12 PM. Reason: Additional information
            http://www.poolnationfx.com/
            http://www.poolnationvrgame.com/

            Comment

            Working...
            X