Unreal Engine 5.1.1 - Pipeline State Object Caching (PSO) Discarded due to Invertibility Fail

Development Platform & Builds
,
1

Using the Meta branch of UE5.1.1 and trying to get PSO caching to work.
I can generate the files on the Quest 2, copy them over, but when I try and expand them all of the shaders from the project (except the pre-made Niagara ones) get discarded.

I edited the engine code so that I could see what is happening; it seems like the invertibility check creates a shader object, uses that object to create a string version of it, and then creates a dummy from that string that it compares to the first.

If I look at the strings that are generated, I notice there is a 16 in the dummy one that is not present in the original.

Since it throws them all out, I still get a lot of hitches in game and I’m not sure what to do.

Is there a bug related to creating or checking the PSO entries?

[2023.04.23-03.00.22:946][ 0]LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: C:\p4v\PDTAMR\PDTAMR\PSOCache/++UE5+Release-5.1-CL-0-PDTAMR_SF_VULKAN_ES31_ANDROID_21986BFD69C444C98A988C156D7F1506.rec.upipelinecache]
[2023.04.23-03.00.22:946][ 0]LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: C:\p4v\PDTAMR\PDTAMR\PSOCache/++UE5+Release-5.1-CL-0-PDTAMR_SF_VULKAN_ES31_ANDROID_21986BFD69C444C98A988C156D7F1506.rec.upipelinecache]
[2023.04.23-03.00.22:946][ 0]LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: C:\p4v\PDTAMR\PDTAMR\PSOCache/++UE5+Release-5.1-CL-0-PDTAMR_SF_VULKAN_ES31_ANDROID_21986BFD69C444C98A988C156D7F1506.rec.upipelinecache]
[2023.04.23-03.00.22:946][ 0]LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: C:\p4v\PDTAMR\PDTAMR\PSOCache/++UE5+Release-5.1-CL-0-PDTAMR_SF_VULKAN_ES31_ANDROID_21986BFD69C444C98A988C156D7F1506.rec.upipelinecache]
[2023.04.23-03.00.22:946][ 0]LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: C:\p4v\PDTAMR\PDTAMR\PSOCache/++UE5+Release-5.1-CL-0-PDTAMR_SF_VULKAN_ES31_ANDROID_21986BFD69C444C98A988C156D7F1506.rec.upipelinecache]
[2023.04.23-03.00.22:946][ 0]LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: C:\p4v\PDTAMR\PDTAMR\PSOCache/++UE5+Release-5.1-CL-0-PDTAMR_SF_VULKAN_ES31_ANDROID_21986BFD69C444C98A988C156D7F1506.rec.upipelinecache]
[2023.04.23-03.00.22:946][ 0]LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: C:\p4v\PDTAMR\PDTAMR\PSOCache/++UE5+Release-5.1-CL-0-PDTAMR_SF_VULKAN_ES31_ANDROID_21986BFD69C444C98A988C156D7F1506.rec.upipelinecache]
[2023.04.23-03.00.22:946][ 0]LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: C:\p4v\PDTAMR\PDTAMR\PSOCache/++UE5+Release-5.1-CL-0-PDTAMR_SF_VULKAN_ES31_ANDROID_21986BFD69C444C98A988C156D7F1506.rec.upipelinecache]
[2023.04.23-03.00.22:946][ 0]LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: C:\p4v\PDTAMR\PDTAMR\PSOCache/++UE5+Release-5.1-CL-0-PDTAMR_SF_VULKAN_ES31_ANDROID_21986BFD69C444C98A988C156D7F1506.rec.upipelinecache]
[2023.04.23-03.00.22:946][ 0]LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: C:\p4v\PDTAMR\PDTAMR\PSOCache/++UE5+Release-5.1-CL-0-PDTAMR_SF_VULKAN_ES31_ANDROID_21986BFD69C444C98A988C156D7F1506.rec.upipelinecache]
[2023.04.23-03.00.22:946][ 0]LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: C:\p4v\PDTAMR\PDTAMR\PSOCache/++UE5+Release-5.1-CL-0-PDTAMR_SF_VULKAN_ES31_ANDROID_21986BFD69C444C98A988C156D7F1506.rec.upipelinecache]
[2023.04.23-03.00.22:946][ 0]LogShaderPipelineCacheTools: Warning: Bad PSO found discarding [Invertibility=FAIL Verify=PASS in: C:\p4v\PDTAMR\PDTAMR\PSOCache/++UE5+Release-5.1-CL-0-PDTAMR_SF_VULKAN_ES31_ANDROID_21986BFD69C444C98A988C156D7F1506.rec.upipelinecache]

I modified the invertibility check to be able to see the string difference:

bool CheckPSOStringInveribility(const FPipelineCacheFileFormatPSO& Item)
{
FPipelineCacheFileFormatPSO TempItem(Item);
TempItem.Hash = 0;

FString StringRep;
switch (Item.Type)
{
case FPipelineCacheFileFormatPSO::DescriptorType::Compute:
	StringRep = TempItem.ComputeDesc.ToString();
	break;
case FPipelineCacheFileFormatPSO::DescriptorType::Graphics:
	StringRep = TempItem.GraphicsDesc.ToString();
	break;
case FPipelineCacheFileFormatPSO::DescriptorType::RayTracing:
	StringRep = TempItem.RayTracingDesc.ToString();
	break;
default:
	return false;
}

FPipelineCacheFileFormatPSO DupItem;
FMemory::Memzero(DupItem.GraphicsDesc);
DupItem.Type = Item.Type;
DupItem.UsageMask = Item.UsageMask;

FString Temp;
switch (Item.Type)
{
case FPipelineCacheFileFormatPSO::DescriptorType::Compute:
	DupItem.ComputeDesc.FromString(StringRep);
	Temp = DupItem.ComputeDesc.ToString();
	break;
case FPipelineCacheFileFormatPSO::DescriptorType::Graphics:
	DupItem.GraphicsDesc.FromString(StringRep);
	Temp = DupItem.GraphicsDesc.ToString();
	break;
case FPipelineCacheFileFormatPSO::DescriptorType::RayTracing:
	DupItem.RayTracingDesc.FromString(StringRep);
	Temp = DupItem.RayTracingDesc.ToString();
	break;
default:
	return false;
}

const bool bFirstCheck = DupItem == TempItem;
if(!bFirstCheck)
{
	UE_LOG(LogShaderPipelineCacheTools, Warning, TEXT("FAIL\n\nItem\n%s\nDupItem\n%s"), *StringRep, *Temp);	
}

return (DupItem == TempItem) && (GetTypeHash(DupItem) == GetTypeHash(TempItem));

}

Here’s what you can see wrong:

Item
23F35A661C129FDA205511F696EF6BACA2AED781,0743E2BFC32A62EE48BD6F8BB8DE108B8D4A7D69,0000000000000000000000000000000000000000,0000000000000000000000000000000000000000,0000000000000000000000000000000000000000,<0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 1 0>,<0.000000 0.000000 2 1 0 1 0>,<1 3 0 7 0 0 0 0 7 0 0 0 255 255>,4,11,4620,2,2,0,0,0,1,37,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,2,0,14,<0 0 3 0 12 0>,<1 0 5 1 8 0>,<1 4 5 2 8 0>,<2 0 12 5 4 0>,<2 0 12 6 4 0>,<2 0 12 7 4 0>,<2 0 12 8 4 0>,<3 0 8 13 0 0>,<4 0 6 3 16 0>,<4 4 6 14 16 0>,<4 8 7 4 16 0>,<4 12 7 15 16 0>,<5 0 3 9 0 0>,<5 0 3 10 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>
DupItem
23F35A661C129FDA205511F696EF6BACA2AED781,0743E2BFC32A62EE48BD6F8BB8DE108B8D4A7D69,0000000000000000000000000000000000000000,0000000000000000000000000000000000000000,0000000000000000000000000000000000000000,<0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 1 0>,<0.000000 0.000000 2 1 0 1 0>,<1 3 0 7 0 0 0 0 7 0 0 0 255 255>,4,11,4620,2,2,0,0,0,1,37,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,2,0,14,<0 0 3 0 12 0>,<1 0 5 1 8 0>,<1 4 5 2 8 0>,<2 0 12 5 4 0>,<2 0 12 6 4 0>,<2 0 12 7 4 0>,<2 0 12 8 4 0>,<3 0 8 13 0 0>,<4 0 6 3 16 0>,<4 4 6 14 16 0>,<4 8 7 4 16 0>,<4 12 7 15 16 0>,<5 0 3 9 0 0>,<5 0 3 10 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>

Doing a diff:

23F35A661C129FDA205511F696EF6BACA2AED781,0743E2BFC32A62EE48BD6F8BB8DE108B8D4A7D69,0000000000000000000000000000000000000000,0000000000000000000000000000000000000000,0000000000000000000000000000000000000000,<0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 1 0>,<0.000000 0.000000 2 1 0 1 0>,<1 3 0 7 0 0 0 0 7 0 0 0 255 255>,4,11,4620,2,2,0,0,0,1,37,160,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,2,0,14,<0 0 3 0 12 0>,<1 0 5 1 8 0>,<1 4 5 2 8 0>,<2 0 12 5 4 0>,<2 0 12 6 4 0>,<2 0 12 7 4 0>,<2 0 12 8 4 0>,<3 0 8 13 0 0>,<4 0 6 3 16 0>,<4 4 6 14 16 0>,<4 8 7 4 16 0>,<4 12 7 15 16 0>,<5 0 3 9 0 0>,<5 0 3 10 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>

There’s a 16 in one and not the other, for every shader.

2

Here’s a human readable version of this shader that didn’t pass:

TempItem:
PSO hash 953313030 mask 18446744073709551615 bindc 7985
VS:23F35A661C129FDA205511F696EF6BACA2AED781 PS:0743E2BFC32A62EE48BD6F8BB8DE108B8D4A7D69
BS:<0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 1 0> RS:<0.000000 0.000000 2 1 0 1 0> DSS:<1 3 0 7 0 0 0 0 7 0 0 0 255 255>
MSAA:4 DSfmt:11 DSflags:4620
DL:2 SL:2 DS:0 SS:0 PT:0
RTA 1
RT0:fmt=37 flg=16
SuH:1 SuI:0
MVC:2 HasFDM:0
NumVE 14
0:<0 0 3 0 12 0> 1:<1 0 5 1 8 0> 2:<1 4 5 2 8 0> 3:<2 0 12 5 4 0> 4:<2 0 12 6 4 0> 5:<2 0 12 7 4 0> 6:<2 0 12 8 4 0> 7:<3 0 8 13 0 0> 8:<4 0 6 3 16 0> 9:<4 4 6 14 16 0> 10:<4 8 7 4 16 0> 11:<4 12 7 15 16 0> 12:<5 0 3 9 0 0> 13:<5 0 3 10 0 0>

[2023.04.23-12.29.17:145][ 0]LogShaderPipelineCacheTools: Warning:
DupItem:
PSO hash 748733660 mask 18446744073709551615 bindc 0
VS:23F35A661C129FDA205511F696EF6BACA2AED781 PS:0743E2BFC32A62EE48BD6F8BB8DE108B8D4A7D69
BS:<0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 1 0> RS:<0.000000 0.000000 2 1 0 1 0> DSS:<1 3 0 7 0 0 0 0 7 0 0 0 255 255>
MSAA:4 DSfmt:11 DSflags:4620
DL:2 SL:2 DS:0 SS:0 PT:0
RTA 1
RT0:fmt=37 flg=0
SuH:1 SuI:0
MVC:2 HasFDM:0
NumVE 14
0:<0 0 3 0 12 0> 1:<1 0 5 1 8 0> 2:<1 4 5 2 8 0> 3:<2 0 12 5 4 0> 4:<2 0 12 6 4 0> 5:<2 0 12 7 4 0> 6:<2 0 12 8 4 0> 7:<3 0 8 13 0 0> 8:<4 0 6 3 16 0> 9:<4 4 6 14 16 0> 10:<4 8 7 4 16 0> 11:<4 12 7 15 16 0> 12:<5 0 3 9 0 0> 13:<5 0 3 10 0 0>

Looks like the only thing that doesn’t agree is “flg=16” and “flg=0”.

This appears to be:
/** Flags used for texture creation */
enum class ETextureCreateFlags : uint64
{
None = 0,

// Texture can be used as a render target
RenderTargetable                  = 1ull << 0,
// Texture can be used as a resolve target
ResolveTargetable                 = 1ull << 1,
// Texture can be used as a depth-stencil target.
DepthStencilTargetable            = 1ull << 2,
// Texture can be used as a shader resource.
ShaderResource                    = 1ull << 3,
// Texture is encoded in sRGB gamma space
SRGB                              = 1ull << 4,
// Texture data is writable by the CPU

Textures is encoded in sRGB gamma space.

3

This is blowing my mind. I added a bunch of logging to the function used to build a shader from string, and it appears that it blows out the RenderTargetFlags when it sets the last one.

Src <0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 0 1 0 0 1 0 15 1 0>,<0.000000 0.000000 2 1 0 1 0>,<1 3 0 7 0 0 0 0 7 0 0 0 255 255>,4,11,4620,2,2,0,0,0,1,37,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,2,0,14,<0 0 3 0 12 0>,<1 0 5 1 8 0>,<1 4 5 2 8 0>,<2 0 12 5 4 0>,<2 0 12 6 4 0>,<2 0 12 7 4 0>,<2 0 12 8 4 0>,<3 0 8 13 0 0>,<4 0 6 3 16 0>,<4 4 6 14 16 0>,<4 8 7 4 16 0>,<4 12 7 15 16 0>,<5 0 3 9 0 0>,<5 0 3 10 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>
PartIt 37,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,2,0,14,<0 0 3 0 12 0>,<1 0 5 1 8 0>,<1 4 5 2 8 0>,<2 0 12 5 4 0>,<2 0 12 6 4 0>,<2 0 12 7 4 0>,<2 0 12 8 4 0>,<3 0 8 13 0 0>,<4 0 6 3 16 0>,<4 4 6 14 16 0>,<4 8 7 4 16 0>,<4 12 7 15 16 0>,<5 0 3 9 0 0>,<5 0 3 10 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>,<0 0 0 0 0 0>
[2023.04.23-14.02.36:226][ 0]LogTemp: Warning: 0 RTF: 16 → 16 (0 = 16)
[2023.04.23-14.02.36:226][ 0]LogTemp: Warning: 0 INTERNAL RTF: 16
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 1 RTF: 0 → 0 (0 = 16)
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 0 INTERNAL RTF: 16
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 1 INTERNAL RTF: 0
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 2 RTF: 0 → 0 (0 = 16)
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 0 INTERNAL RTF: 16
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 1 INTERNAL RTF: 0
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 2 INTERNAL RTF: 0
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 3 RTF: 0 → 0 (0 = 16)
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 0 INTERNAL RTF: 16
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 1 INTERNAL RTF: 0
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 2 INTERNAL RTF: 0
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 3 INTERNAL RTF: 0
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 4 RTF: 0 → 0 (0 = 16)
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 0 INTERNAL RTF: 16
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 1 INTERNAL RTF: 0
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 2 INTERNAL RTF: 0
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 3 INTERNAL RTF: 0
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 4 INTERNAL RTF: 0
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 5 RTF: 0 → 0 (0 = 16)
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 0 INTERNAL RTF: 16
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 1 INTERNAL RTF: 0
[2023.04.23-14.02.36:227][ 0]LogTemp: Warning: 2 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 3 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 4 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 5 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 6 RTF: 0 → 0 (0 = 16)
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 0 INTERNAL RTF: 16
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 1 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 2 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 3 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 4 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 5 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 6 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 7 RTF: 0 → 0 (0 = 0)
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 0 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 1 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 2 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 3 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 4 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 5 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 6 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 7 INTERNAL RTF: 0
[2023.04.23-14.02.36:228][ 0]LogTemp: Warning: 0 AFTER CHECK RTF: 0
[2023.04.23-14.02.36:229][ 0]LogTemp: Warning: 1 AFTER CHECK RTF: 0
[2023.04.23-14.02.36:229][ 0]LogTemp: Warning: 2 AFTER CHECK RTF: 0
[2023.04.23-14.02.36:229][ 0]LogTemp: Warning: 3 AFTER CHECK RTF: 0
[2023.04.23-14.02.36:229][ 0]LogTemp: Warning: 4 AFTER CHECK RTF: 0
[2023.04.23-14.02.36:229][ 0]LogTemp: Warning: 5 AFTER CHECK RTF: 0
[2023.04.23-14.02.36:229][ 0]LogTemp: Warning: 6 AFTER CHECK RTF: 0
[2023.04.23-14.02.36:229][ 0]LogTemp: Warning: 7 AFTER CHECK RTF: 0

for (int32 Index = 0; Index < MaxSimultaneousRenderTargets; Index++)
{
check(PartEnd - PartIt >= 4); //not a very robust parser
LexFromString((uint32&)(RenderTargetFormats[Index]), *PartIt++);

	ETextureCreateFlags RTFlags;
	LexFromString(RTFlags, *PartIt++);
	
	// going forward, the flags will already be reduced when logging the PSOs to disk. However as of 2021-06-17 there are still old stable cache files in existence that have flags recorded as is
	RenderTargetFlags[Index] = ReduceRTFlags(RTFlags);

	UE_LOG(LogTemp, Warning, TEXT("%d RTF: %lld -> %lld (0 = %lld)"), Index, RTFlags, RenderTargetFlags[Index], RenderTargetFlags[0]);
	
	uint8 Load, Store;
	LexFromString(Load, *PartIt++);
	LexFromString(Store, *PartIt++);

	for(int32 InnerIndex = 0; InnerIndex <= Index; ++InnerIndex)
	{
		UE_LOG(LogTemp, Warning, TEXT("%d INTERNAL RTF: %lld"), InnerIndex, RenderTargetFlags[InnerIndex]);
	}	
}

for(int32 Index = 0; Index < MaxSimultaneousRenderTargets; ++Index)
{
	UE_LOG(LogTemp, Warning, TEXT("%d AFTER CHECK RTF: %lld"), Index, RenderTargetFlags[Index]);
}
4

There is a bug in line 677 of PipelineFileCache.cpp, which is causing this problem.

for (int32 Index = 0; Index < MaxSimultaneousRenderTargets; Index++)
{
check(PartEnd - PartIt >= 4); //not a very robust parser
LexFromString((uint32&)(RenderTargetFormats[Index]), *PartIt++);
ETextureCreateFlags RTFlags;
LexFromString(RTFlags, *PartIt++);
// going forward, the flags will already be reduced when logging the PSOs to disk. However as of 2021-06-17 there are still old stable cache files in existence that have flags recorded as is
RenderTargetFlags[Index] = ReduceRTFlags(RTFlags);
uint8 Load, Store;
LexFromString(Load, *PartIt++);
LexFromString(Store, *PartIt++);
}

The LexFromString((uint32&) is stamping on the RenderTargetFlags array.

Here’s what will fix it:

bool FPipelineCacheFileFormatPSO::GraphicsDescriptor::StateFromString(const FStringView& Src)
{
static_assert(sizeof(EPixelFormat) == 1);
static_assert(sizeof(ERenderTargetLoadAction) == 1);
static_assert(sizeof(ERenderTargetStoreAction) == 1);
static_assert(sizeof(DepthLoad) == 1);
static_assert(sizeof(DepthStore) == 1);
static_assert(sizeof(StencilLoad) == 1);
static_assert(sizeof(StencilStore) == 1);
static_assert(sizeof(PrimitiveType) == 4);

constexpr int32 PartCount = FPipelineCacheGraphicsDescPartsNum;

TArray<FStringView, TInlineAllocator<PartCount>> Parts;
UE::String::ParseTokens(Src.TrimStartAndEnd(), TEXT(','), [&Parts](FStringView Part) { Parts.Add(Part); });

// check if we have expected number of parts
if (Parts.Num() != PartCount)
{
	// instead of crashing let caller handle this case
	return false;
}

const FStringView* PartIt = Parts.GetData();
const FStringView* PartEnd = PartIt + PartCount;

check(PartEnd - PartIt >= 3); //not a very robust parser
BlendState.FromString(*PartIt++);
RasterizerState.FromString(*PartIt++);
DepthStencilState.FromString(*PartIt++);

check(PartEnd - PartIt >= 3); //not a very robust parser
LexFromString(MSAASamples, *PartIt++);
LexFromString((uint32&)DepthStencilFormat, *PartIt++);
LexFromString(DepthStencilFlags, *PartIt++);

check(PartEnd - PartIt >= 5); //not a very robust parser
LexFromString((uint32&)DepthLoad, *PartIt++);
LexFromString((uint32&)StencilLoad, *PartIt++);
LexFromString((uint32&)DepthStore, *PartIt++);
LexFromString((uint32&)StencilStore, *PartIt++);
LexFromString((uint32&)PrimitiveType, *PartIt++);

check(PartEnd - PartIt >= 1); //not a very robust parser
LexFromString(RenderTargetsActive, *PartIt++);

for (int32 Index = 0; Index < MaxSimultaneousRenderTargets; Index++)
{
	check(PartEnd - PartIt >= 4); //not a very robust parser

	// START ENGINE CHANGE

	// OLD LINE:
	//LexFromString((uint32&)(RenderTargetFormats[Index]), *PartIt++);

	// This causes the RenderTargetFlags array to get stomped, since the RenderTargetFormats are not uint32.
	
	// NEW CODE:
	uint8 RenderTargetFormat;
	LexFromString(RenderTargetFormat, *PartIt++);
	RenderTargetFormats[Index] = static_cast<EPixelFormat>(RenderTargetFormat);

	// END ENGINE CHANGE
	
	ETextureCreateFlags RTFlags;
	LexFromString(RTFlags, *PartIt++);
	// going forward, the flags will already be reduced when logging the PSOs to disk. However as of 2021-06-17 there are still old stable cache files in existence that have flags recorded as is
	RenderTargetFlags[Index] = ReduceRTFlags(RTFlags);
	uint8 Load, Store;
	LexFromString(Load, *PartIt++);
	LexFromString(Store, *PartIt++);
}

// parse sub-pass information
{
	uint32 LocalSubpassHint = 0;
	uint32 LocalSubpassIndex = 0;
	check(PartEnd - PartIt >= 2);
	LexFromString(LocalSubpassHint, *PartIt++);
	LexFromString(LocalSubpassIndex, *PartIt++);
	SubpassHint = LocalSubpassHint;
	SubpassIndex = LocalSubpassIndex;
}

// parse multiview and FDM information
{
	uint32 LocalMultiViewCount = 0;
	uint32 LocalHasFDM = 0;
	check(PartEnd - PartIt >= 2);
	LexFromString(LocalMultiViewCount, *PartIt++);
	LexFromString(LocalHasFDM, *PartIt++);
	MultiViewCount = (uint8)LocalMultiViewCount;
	bHasFragmentDensityAttachment = (bool)LocalHasFDM;
}

check(PartEnd - PartIt >= 1); //not a very robust parser
int32 VertDescNum = 0;
LexFromString(VertDescNum, *PartIt++);
check(VertDescNum >= 0 && VertDescNum <= MaxVertexElementCount);

VertexDescriptor.Empty(VertDescNum);
VertexDescriptor.AddZeroed(VertDescNum);

check(PartEnd - PartIt == MaxVertexElementCount); //not a very robust parser
for (int32 Index = 0; Index < VertDescNum; Index++)
{
	VertexDescriptor[Index].FromString(*PartIt++);
}

check(PartIt + MaxVertexElementCount == PartEnd + VertDescNum);

VertexDescriptor.Sort([](FVertexElement const& A, FVertexElement const& B)
  {
	  if (A.StreamIndex < B.StreamIndex)
	  {
		  return true;
	  }
	  if (A.StreamIndex > B.StreamIndex)
	  {
		  return false;
	  }
	  if (A.Offset < B.Offset)
	  {
		  return true;
	  }
	  if (A.Offset > B.Offset)
	  {
		  return false;
	  }
	  if (A.AttributeIndex < B.AttributeIndex)
	  {
		  return true;
	  }
	  if (A.AttributeIndex > B.AttributeIndex)
	  {
		  return false;
	  }
	  return false;
  });

return true;

}

5

This is fixed in the latest version of 5.1.1, it’s that I’m using an older Meta version.

6

Please help. Earlier I have done successfully PSO caching , in 4.27.2, but from 5.0 and in 5.2 final file is too small 20 KB , earlier was over 1MB .Now I have hitches in my game but earlier in 4.27 with PSO worked perfectly.
What is wrong with PSO in 5.1.1,5.2? I have info that uses PSO but only small number of shaders and has no effect on lags in the game.

Please help, I would use 4.27.2 and leave Target API 29 but Google asks for target API 33 for new updates and and only API 31 will be visible on Android 11+ devices.

Thanks in advance.

7

You should look at the results when you run the utility that creates the PSO output and see what it says. It tells you how many objects it uses and how many it throws out and what the problem is. My problem is fixed with the meta branch 5.1, I can verify PSO caching works. I haven’t tried vanilla UE5.1 or 5.2.