I’ve the same assert at PipelineCacheUtilities.cpp:205. My config: Win64 | D3D12 | PCD3D_SM6, 5.8.1 CL 56057345
I made a research looking for why 1aydan’s fix works.
Two caveats up front: this is static analysis against the installed 5.8.1 source because I have not been able to build a patched engine to confirm it. And all line numbers below are from the 5.8.1 release, so they will differ on ue5-main branch.
1. File that identifies a shader hash by the size of the write
PipelineCacheUtilities.cpp:137
/**
* FShaderHash values are serialized like a binary stream, so assume every serialization of this size is a FShaderHash.
*/
virtual void Serialize(void* V, int64 Length) override
{
if (Length == sizeof(FShaderHash))
{
// ...treat as a hash: read and write a deduplicated index
2. Size changed from 20 bytes to 8 between 5.7 and 5.8
|
5.7 |
5.8 |
| Type |
FSHAHash > alignas(uint32) uint8 Hash[20] (Misc/SecureHash.h:225) |
FShaderHash : FXxHash64 > a single uint64 (Hash/ShaderHash.h:11-17) |
sizeof |
20 |
8 |
With 20 bytes, the heuristic was reliable because no stream wrote exactly that amount. Now, with 8 bytes, it causes collisions, as that is one of the most common chunk sizes.
3. Varints are chunked differently on the write and read paths
Runtime/Core/Private/Serialization/VarInt.cpp
// write: one call
void WriteVarUIntToArchive(FArchive& Ar, uint64 Value)
{
uint8 Buffer[9];
const uint32 Size = WriteVarUInt(Value, Buffer);
Ar.Serialize(Buffer, Size);
}
// read: two calls
uint64 ReadVarUIntFromArchive(FArchive& Ar)
{
uint8 Buffer[9];
Ar.Serialize(Buffer, 1);
uint32 Size = MeasureVarUInt(Buffer);
if (Size > 1)
Ar.Serialize(Buffer + 1, Size - 1);
return ReadVarUInt(Buffer, Size);
}
So for a value whose varint encoding is 9 bytes:
|
bytes passed to Serialize() |
intercepted as a hash? |
| write |
Serialize(Buffer, 9) |
no > 9 != 8 |
| read |
Serialize(Buffer, 1), then Serialize(Buffer + 1, 8) |
yes, on the second call |
9-byte varints represent the worst-case scenario in a specific sense: the initial byte is 0xFF and contains no useful data bits, so the remaining 8 bytes constitute the raw uint64 exactly sizeof(FShaderHash).
There is a reverse case: a value whose varint representation occupies 8 bytes (2^49 <= v < 2^56) fails in the opposite way, it is intercepted upon writing but read in raw form. Any value encoded in 7 bytes or fewer is symmetric and safe.
That asymmetry creates a lack of synchronization and explains why bypassing the proxy at those two call sites resolves the issue. I would add that the proxy already performs exactly this action internally: when it needs to write its own varints, it sends them directly to InnerArchive (lines 147, 180, and 184), precisely to avoid intercepting itself.
4. Why the assertion appears late and with different signatures
In the loading loop, LoadActiveSlots executes first (line 791) before the PSO is read (797) and before the varint fields (799–800) with the check occurring on line 804. This means that a synchronization mismatch introduced while reading permutation group N does not cause a failure at that point, but rather when group N+1 reads its active slots.
Depending on the content of the misaligned bytes, this manifests as the ActivePerSlot assertion, a checkNoEntry, a TArray resizing failure, or the “impossible stable shader key index” checkf statement on line 290. This aligns with Fri_Me_Dev’s observation that the signature varies by version and platform: it is the same synchronization issue manifesting at different points.
What I could not confirm
Which field actually holds the excessive value? I initially assumed it was NewPso.UsageMask = uint64(-1) (ShaderPipelineCacheToolsCommandlet.cpp:2224), but that is incorrect: AddComputePSOs is only called from BuildPSOSC at line 2679 that is, after the call to LoadStablePipelineCacheFile at line 2575 (where the assertion failure occurs) so that value never makes it into the .spc file. The usage masks that do end up in the file come from the recording, where the default value is 0 (GameUsageMask is 0 unless the title calls SetGameUsageMaskWithComparison).
Thus, the mechanism described explains how the file desynchronization can occur and why the fix affects those two lines, but I have not yet determined exactly which specific value triggers the issue. The remaining candidates are the other points where varint is called; the static_cast<uint64> of an index at lines 510/512 seems the most likely culprit. If a source build is available, logging the Length value and the calling field within that Serialize overload should make it possible to identify the specific instance in a single run.
On the fix itself
The two-line change appears structurally correct but is limited in scope. The other varint call sites in the same file (599/800, 602/807, 510/512, 277/289) are safe only because their values happen to be small not by design, and without any validation to guarantee it. Furthermore, the heuristic is not specific to varints: any 8-byte raw data using Serialize within that file is interpreted as a hash upon reading.
A robust solution would be to stop identifying hashes based on the Serialize length and instead provide FShaderHash with an explicit serialization path; this way, future modifications to the type would not reintroduce the same vulnerability.
### Things that didn’t work for me, in case this happens to anyone else.
- Re-recording the PSOs doesn’t help: I tried with two separate recordings, and both failed.
- Deleting derived data doesn’t solve it either: I tried deleting the DDC, the Zen store, and the cooked output, but the same assertion failure occurs.
