Pak forward compatibility

bove.san · December 23, 2021, 1:15pm

After upgrading our project from 4.26 to 4.27, I can no longer load .pak files created with 4.26. Cooking and packing the same assets with 4.27 creates .pak compatible with the upgraded project. I’m under the impression that assets should be forward compatible, and I’ve tried setting the Changelist property in the project from 0 to 18319896 to match the official 4.27 release, after reading this:

The crash happens when asserting the compression blocks, but disabling compression just yields a different assertion error.

D/UE4     : [2021.12.23-08.46.23:869][257]LogPakFile: PakFile PrimaryIndexSize=1822
D/UE4     : [2021.12.23-08.46.23:870][257]LogPakFile: PakFile PathHashIndexSize=404
D/UE4     : [2021.12.23-08.46.23:871][257]LogPakFile: PakFile FullDirectoryIndexSize=1662
D/UE4     : [2021.12.23-08.46.23:874][257]LogShaderLibrary: Display: ShaderCodeLibraryPakFileMountedCallback: PakFile '/storage/emulated/0/Android/data/com.xxxxxxxx.xxxxxxxx/files/Paks/4BQ1xc2Q8rw1NC4uvrFKTV-Android_ASTC.pak' (chunk index -1, root '../../../') mounted
D/UE4     : [2021.12.23-08.46.23:875][257]LogShaderLibrary: Display: ShaderCodeLibraryPakFileMountedCallback: pending pak file info (ChunkID:-1 Root:../../../ File:/storage/emulated/0/Android/data/com.xxxxxxxx.xxxxxxxx/files/Paks/4BQ1xc2Q8rw1NC4uvrFKTV-Android_ASTC.pak)
D/UE4     : [2021.12.23-08.46.23:875][257]LogPakFile: OnPakFileMounted2Time == 0.004588
D/UE4     : [2021.12.23-08.46.23:877][257]LogXxxxxxxxEvents: Display: EXxxxxxxxEvent::Event Updated access time for 4BQ1xc2Q8rw1NC4uvrFKTV-Android_ASTC
D/UE4     : [2021.12.23-08.46.23:880][257]LogGameMode: ProcessServerTravel: 4BQ1xc2Q8rw1NC4uvrFKTV
D/UE4     : [2021.12.23-08.46.23:953][257]LogPackageName: SearchForPackageOnDisk took   0.067s to resolve 4BQ1xc2Q8rw1NC4uvrFKTV.umap.
D/UE4     : [2021.12.23-08.46.23:962][257]LogOutputDevice: Warning: 
D/UE4     : 
D/UE4     : Script Stack (8 frames):
D/UE4     : MainMenu_C.ExecuteUbergraph_MainMenu
D/UE4     : MainMenu_C.OnMapSelected
D/UE4     : Maps_C.ExecuteUbergraph_Maps
D/UE4     : Maps_C.OnMapSelect
D/UE4     : MapItem_C.ExecuteUbergraph_MapItem
D/UE4     : MapItem_C.BndEvt__ListItem_K2Node_ComponentBoundEvent_0_OnClick__DelegateSignature
D/UE4     : ListItem_C.ExecuteUbergraph_ListItem
D/UE4     : ListItem_C.BndEvt__Button_K2Node_ComponentBoundEvent_0_OnButtonClickedEvent__DelegateSignature
D/UE4     : 
D/UE4     : Assertion failed: (UncompressedFileSize + CompressionBlockSize - 1) / CompressionBlockSize == FileEntry.CompressionBlocks.Num() [File:C:/xxxxxxxx/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp] [Line: 3828] 
D/UE4     : 
D/UE4     : [2021.12.23-08.46.24:665][257]Assertion failed: (UncompressedFileSize + CompressionBlockSize - 1) / CompressionBlockSize == FileEntry.CompressionBlocks.Num() [File:C:/xxxxxxxx/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp] [Line: 3828] 
D/UE4     : 
D/UE4     : [2021.12.23-08.46.24:665][257]LogAndroid: Error: === Critical error: ===
D/UE4     : [2021.12.23-08.46.24:666][257]LogAndroid: Error: 
D/UE4     : [2021.12.23-08.46.24:667][257]LogAndroid: Error: Assertion failed: (UncompressedFileSize + CompressionBlockSize - 1) / CompressionBlockSize == FileEntry.CompressionBlocks.Num() [File:C:/xxxxxxxx/Engine/Source/Runtime/PakFile/Private/IPlatformFilePak.cpp] [Line: 3828] 
D/UE4     : [2021.12.23-08.46.24:667][257]LogAndroid: Error: 
D/UE4     : [2021.12.23-08.46.24:667][257]LogAndroid: Error: << callstack too long >>
D/UE4     : [2021.12.23-08.46.24:667][257]LogAndroid: Error: 
D/UE4     : [2021.12.23-08.46.24:667][257]LogAndroid: Error: 
D/UE4     : [2021.12.23-08.46.24:676][257]LogExit: Executing StaticShutdownAfterError
D/UE4     : [2021.12.23-08.46.24:687][257]LogAndroid: FAndroidMisc::RequestExit(1)

bove.san · December 28, 2021, 9:41am

Attempting to verify the 4.26 pak with UnrealPak from 4.27 yields the following:

PS C:\Users\xxxxxxxx\git\p4-xxxxxxxx\ue\Engine\Binaries\Win64> .\UnrealPak.exe C:\Users\xxxxxxxx\AppData\Local\Temp\4BQ1xc2Q8rw1NC4uvrFKTV.cloud\4BQ1xc2Q8rw1NC4uvrFKTV-WindowsNoEditor.pak -Verify
LogInit: Display: Loading text-based GConfig....
LogPakFile: Display: Using command line for crypto configuration
LogPakFile: Display: Checking pak file "C:\Users\xxxxxxxx\AppData\Local\Temp\4BQ1xc2Q8rw1NC4uvrFKTV.cloud\4BQ1xc2Q8rw1NC4uvrFKTV-WindowsNoEditor.pak". This may take a while...
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Engine/Content/BasicShapes/Cube.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Engine/Content/BasicShapes/Sphere.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Engine/Content/BasicShapes/Cube.uexp".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Engine/Content/EngineMaterials/DefaultMaterial.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Engine/Content/EngineMaterials/T_Default_Material_Grid_M.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Engine/Content/EngineMaterials/T_Default_Material_Grid_N.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Engine/Content/EngineMaterials/WorldGridMaterial.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Engine/Content/EngineMaterials/T_Default_Material_Grid_M.uexp".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Engine/Content/EngineMaterials/T_Default_Material_Grid_N.uexp".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Engine/Content/Functions/Engine_MaterialFunctions01/Opacity/CameraDepthFade.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Engine/Content/Functions/Engine_MaterialFunctions01/Opacity/CameraDepthFade.uexp".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Xxxxxxxx/Content/4BQ1xc2Q8rw1NC4uvrFKTV/Maps/HeadOn/4BQ1xc2Q8rw1NC4uvrFKTV.umap".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Xxxxxxxx/Content/4BQ1xc2Q8rw1NC4uvrFKTV/Maps/HeadOn/4BQ1xc2Q8rw1NC4uvrFKTV_BuiltData.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Xxxxxxxx/Content/4BQ1xc2Q8rw1NC4uvrFKTV/Maps/HeadOn/Ambience__Urban__City__Park__Birds__Wind__Distant_Traffic_SND0017.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Xxxxxxxx/Content/4BQ1xc2Q8rw1NC4uvrFKTV/Maps/HeadOn/Blue.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Xxxxxxxx/Content/4BQ1xc2Q8rw1NC4uvrFKTV/Maps/HeadOn/HeadOn4.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Xxxxxxxx/Content/4BQ1xc2Q8rw1NC4uvrFKTV/Maps/HeadOn/Red.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Xxxxxxxx/Content/4BQ1xc2Q8rw1NC4uvrFKTV/Maps/HeadOn/WorldGridMaterial.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Xxxxxxxx/Content/4BQ1xc2Q8rw1NC4uvrFKTV/Maps/HeadOn/4BQ1xc2Q8rw1NC4uvrFKTV.uexp".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Xxxxxxxx/Content/4BQ1xc2Q8rw1NC4uvrFKTV/Maps/HeadOn/Red.uexp".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Xxxxxxxx/Content/4BQ1xc2Q8rw1NC4uvrFKTV/Maps/HeadOn/WorldGridMaterial.uexp".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Xxxxxxxx/Content/4BQ1xc2Q8rw1NC4uvrFKTV/Materials/Sky.uasset".
LogPakFile: Error: Index FPakEntry does not match Payload FPakEntry for "Xxxxxxxx/Content/4BQ1xc2Q8rw1NC4uvrFKTV/Materials/Sky.uexp".
LogPakFile: Display: Pak file "C:\Users\xxxxxxxx\AppData\Local\Temp\4BQ1xc2Q8rw1NC4uvrFKTV.cloud\4BQ1xc2Q8rw1NC4uvrFKTV-WindowsNoEditor.pak" corrupted (23 errors out of 33 files checked.).
LogPakFile: Display: Pak file "C:\Users\xxxxxxxx\AppData\Local\Temp\4BQ1xc2Q8rw1NC4uvrFKTV.cloud\4BQ1xc2Q8rw1NC4uvrFKTV-WindowsNoEditor.pak" checked in 0.01s
LogPakFile: Display: Unreal pak executed in 0.014866 seconds