Pak File Loader Plugin

[Duwayne]

Hello, i hope this has a simple answer.

So i am using this to enable custom maps in the game. if i hard code things it all works fine. loading the pak, registering the asset. and loading the level.

but the one thing i can’t figure out that i want to be able to dynamically do is how do i read any level files in a pak?

for instance:

CustomMapDIR/Map1.Pak -> has Level1.umap
CustomMapDIR/Map2.Pak -> has SweetLevel.umap

how do i read the level1.umap and sweetlevel.umap via blueprints. (aka i dont know the map name ahead of time).

EDIT: NVM i figured it out.

For info for anyone else looking for this.

Register Mount Point: (yes first)
Root: /CustomMap/
ContentPath: …/…/…/CustomMapProject/Plugins/CustomMap/Content/
(make all custom maps with this layout, i made MapName.umap + MapNameContent (folder))

Get Files in Pak Directory:
Pak Dir: C:\CustomMap (variable for location)
Recursive (your choice)

Get Files in Pak Directory output into for each loop:
Valid Pak file? (good practice)

Mount Pak File:
Link Array Element here
Mount Path “” (Leave empty)

Load Pak Asset Registry File:
Asset Registry File: …/…/…/CustomMapProject/Plugins/CustomMap/AssetRegistry.bin

Then w/e you are building your list of custom maps.

Get Asset Registry -> Get Assets by Class
Class Name “World”

into for each loop

Break Array element -> AssetData

You get package, asset name and path to put into array to call later for building map list.

[Duwayne]

Can you make it so we can load a AssetRegistry.bin out of a specific PAK if we know the expected file dir.

like

PAK -> Load Pak Asset Registry specific PAK File

→ PAK File
→ DIR “…/…/…/Project/Plugins/Map/AssetRegistry.bin”

I have a issue if 10 maps/plugins are made with the same layout but different file names under the same dir

in this example say:
PAK1 = “…/…/…/Project/Plugins/Map/Map1.level”
PAK2 = “…/…/…/Project/Plugins/Map/Map2.level”
PAK3 = “…/…/…/Project/Plugins/Map/Map3.level”

it will reload the assetregistry.bin file from the first PAK file over and over. and Map1 will be registered 3 times.

EDIT:

turns out this isn’t really required as it was my own workflow that was the issue. make mount paths unique to each pak file.
then grab the mount point and pass the registry file.

[BlueMountainsIO]

@Duwayne
As far as I know the engine doesn’t load the asset registry file automatically?

There is a function named LoadPakAssetRegistryFile (see documentation).

[JakeTee]

@BlueMountainsIO Hey, I wrote a question on marketplace for you but I thought Im just going to rewrite it here so it’s easier to notice then.
So what I am trying to ask is that, does the plugin handle mounting/loading pak from a plugin that is not registered in the base project. Say, another user create a plugin from the base project, package the plugin and then send the pak to me. Would this Pak Loader be able to load that pak in my base project (that doesn’t contain the user’s plugin)? Similiar to how community work or community maps work. Thanks a lot

[BlueMountainsIO]

@JakeTee

@BlueMountainsIO Hey, I wrote a question on marketplace for you but I thought Im just going to rewrite it here so it’s easier to notice then.
So what I am trying to ask is that, does the plugin handle mounting/loading pak from a plugin that is not registered in the base project. Say, another user create a plugin from the base project, package the plugin and then send the pak to me. Would this Pak Loader be able to load that pak in my base project (that doesn’t contain the user’s plugin)? Similiar to how community work or community maps work. Thanks a lot

The pak mounting procedure is independent from what contents are inside the pak or from which project it originates.
If the mounting succeeds can depend on the engine version where you are loading it from and the engine version where the pak was created. There are also some settings which can interfere with it.

If the content from the pak can be used within your project where it loads from can also depend on the version and on some engine settings. In the end it’s up to Unreal. This plugin just wraps functions around the engine’s one, so that it’s easier to use.

Limitations:

• Only tested on Windows x64.
• Loading assets from mounted Pak files only works in packaged/shipping builds of your project. (Not in Editor)
• You can’t load unsigned Pak files if your game project has Pak signing enabled.
• Loading pak files that have been created with a different engine version than your project might not load.

[Brannydonowt (Branny)]

@BlueMountainsIO

Hi there, first off - great work on the plugin, it seems quite straightforward and I’ve almost got it working for what I need!

I’m getting an error when loading a UMaterialInstance from a Pak that is already mounted, and I’m not too sure if it’s something that you may have encountered before and would be able to point me in the right direction of fixing.

As I’m loading in the Material I get the error "Assertion Failed: ShaderCode.Num() [File:Runtime\RenderCore\Public\ShaderCore.h] [Line: 865]

I’ve created the material that is being applied purely in the Plugin Content folder, so I’m sure there shouldn’t be any crossing over between the two directories.

Any sort of insight would be appreciated.

Regards,

Brandon

[BlueMountainsIO]

@Brannydonowt
Glad it’s useful for you.

This error comes from the engine. Some other user had it as well. I think he solved it by disabling “Share Material Shader Code” in the project settings.

[Brannydonowt (Branny)]

@BlueMountainsIO

@Brannydonowt
Glad it’s useful for you.

This error comes from the engine. Some other user had it as well. I think he solved it by disabling “Share Material Shader Code” in the project settings.

@BlueMountainsIO This fix worked perfectly, thanks!

I have a couple of questions regarding downloading my Pak’s over HTTP.

When generating my Pak File, an accompanying .sig file is created alongside it. It seems as though this .sig file is required in order to properly mount the Pak file to the engine. Is there any way you would recommend this is handled, as I’m aware that there isn’t an included method that would download the .sig file too.

Are there any changes in the packaging process that have to be made in order to properly download and mount a .pak file? I am very nearly there and I know it!

regards,

Brandon

[sivan]

@BlueMountainsIO

Hi,
I just bought your plugin and testing the 4.24 demo project. After rewriting the 2 file names it packages fine, and the PakDLC.log also shows it’s okay, but running the exe gives these 2 errors:

the log:

[2020.04.22-12.46.24] Log: true
[2020.04.22-12.46.24] Verbose: Found PakPlatformFile
[2020.04.22-12.46.24] Log: C:\UE4\PakLoaderPlugin\PakDemoProject\TestDLC.pak Pak OK
[2020.04.22-12.46.24] Log: true
[2020.04.22-12.46.24] Log: …/…/…/PakDemoProject/Content/Paks/PakDemoProject-WindowsNoEditor.pak
[2020.04.22-12.46.24] Log: C:\UE4\PakLoaderPlugin\PakDemoProject\TestDLC.pak
[2020.04.22-12.46.24] Log:

[2020.04.22-12.46.24] Log: …/…/…/TestProject/Plugins/TestDLC/AssetRegistry.bin
[2020.04.22-12.46.24] Log: /TestDLC/Audio/Fire01
[2020.04.22-12.46.24] Log: /TestDLC/Audio/Fire01_Cue
[2020.04.22-12.46.24] Log: /TestDLC/Blueprints/BP_Test
[2020.04.22-12.46.24] Log: /TestDLC/Maps/DLCMap
[2020.04.22-12.46.24] Log: /TestDLC/Materials/M_Chair
[2020.04.22-12.46.24] Log: /TestDLC/Materials/M_Color
[2020.04.22-12.46.24] Log: /TestDLC/Materials/MI_Table
[2020.04.22-12.46.24] Log: /TestDLC/Meshes/SM_Chair
[2020.04.22-12.46.24] Log: /TestDLC/Meshes/SM_Table
[2020.04.22-12.46.24] Log: /TestDLC/Textures/T_Chair_M
[2020.04.22-12.46.24] Log: /TestDLC/Textures/T_Chair_N
[2020.04.22-12.46.24] Log: /TestDLC/Textures/T_TableRound_M
[2020.04.22-12.46.24] Log: /TestDLC/Textures/T_TableRound_N
[2020.04.22-12.46.24] Log: …/…/…/TestProject/Plugins/TestDLC/Metadata/DevelopmentAssetRegistry.bin
[2020.04.22-12.46.24] Log: true
[2020.04.22-12.46.24] Log: Populated pak assets to Unreal’s AssetRegistry
[2020.04.22-12.46.24] Log: /TestDLC/Audio/Fire01_Cue
[2020.04.22-12.46.24] Log: true

[sivan]

so it basically cannot spawn the referenced assets. I made some further tests with a new project (building a shipping game and a DLC pak from another project), and the same happens, the pak is mounted, asset registry can find the assets, but they do not work for spawning.

[AdamHorvath]

Hi Sivan,

After mounting the .pak file you have to load all the assets using the asset registry. You can use the “load asset blocking node” for example to do this.
Only after that, you can dynamic cast the loaded asset and spawn.

EDIT: Also make sure to turn off .pak encryption.

I am using this plugin for a while, and it works like charm.

@sivan

so it basically cannot spawn the referenced assets. I made some further tests with a new project (building a shipping game and a DLC pak from another project), and the same happens, the pak is mounted, asset registry can find the assets, but they do not work for spawning.

[sivan]

thanks I will try that. but the example should work imo, I will check its source…
(earlier I made some trials with pak loading, and there is way I can use pak files copied beside the game pak in 4.20, but not in 4.24)

[sivan]

@BlueMountainsIO

in 4.24 apparently the DLCpak loading fails at BP_Test.
I made a check via Assetregistry, and trying to load the assets by this code:

and the resulting extended log:
[SPOILER]
[2020.04.23-12.09.40] Log: true
[2020.04.23-12.09.40] Verbose: Found PakPlatformFile
[2020.04.23-12.09.40] Log: C:\UE4\PakLoader\Project\TestDLC.pak Pak OK
[2020.04.23-12.09.40] Log: true
[2020.04.23-12.09.40] Log: …/…/…/PakDemoProject/Content/Paks/PakDemoProject-WindowsNoEditor.pak
[2020.04.23-12.09.40] Log: C:\UE4\PakLoader\Project\TestDLC.pak
[2020.04.23-12.09.40] Log:

[2020.04.23-12.09.40] Log: …/…/…/TestProject/Plugins/TestDLC/AssetRegistry.bin
[2020.04.23-12.09.40] Log: /TestDLC/Audio/Fire01
[2020.04.23-12.09.40] Log: /TestDLC/Audio/Fire01_Cue
[2020.04.23-12.09.40] Log: /TestDLC/Blueprints/BP_Test
[2020.04.23-12.09.40] Log: /TestDLC/Maps/DLCMap
[2020.04.23-12.09.40] Log: /TestDLC/Materials/M_Chair
[2020.04.23-12.09.40] Log: /TestDLC/Materials/M_Color
[2020.04.23-12.09.40] Log: /TestDLC/Materials/MI_Table
[2020.04.23-12.09.40] Log: /TestDLC/Meshes/SM_Chair
[2020.04.23-12.09.40] Log: /TestDLC/Meshes/SM_Table
[2020.04.23-12.09.40] Log: /TestDLC/Textures/T_Chair_M
[2020.04.23-12.09.40] Log: /TestDLC/Textures/T_Chair_N
[2020.04.23-12.09.40] Log: /TestDLC/Textures/T_TableRound_M
[2020.04.23-12.09.40] Log: /TestDLC/Textures/T_TableRound_N
[2020.04.23-12.09.40] Log: …/…/…/TestProject/Plugins/TestDLC/Metadata/DevelopmentAssetRegistry.bin
[2020.04.23-12.09.40] Log: true
[2020.04.23-12.09.40] Log: Populated pak assets to Unreal’s AssetRegistry
[2020.04.23-12.09.40] Log: Object Path : /TestDLC/Audio/Fire01.Fire01
[2020.04.23-12.09.40] Log: Package Name : /TestDLC/Audio/Fire01
[2020.04.23-12.09.40] Log: Package Path : /TestDLC/Audio
[2020.04.23-12.09.40] Log: Asset Name : Fire01
[2020.04.23-12.09.40] Log: Asset Class : SoundWave
[2020.04.23-12.09.40] Log: Object Path : /TestDLC/Audio/Fire01_Cue.Fire01_Cue
[2020.04.23-12.09.40] Log: Package Name : /TestDLC/Audio/Fire01_Cue
[2020.04.23-12.09.40] Log: Package Path : /TestDLC/Audio
[2020.04.23-12.09.40] Log: Asset Name : Fire01_Cue
[2020.04.23-12.09.40] Log: Asset Class : SoundCue
[2020.04.23-12.09.40] Log: Object Path : /TestDLC/Blueprints/BP_Test.BP_Test
[2020.04.23-12.09.40] Log: Package Name : /TestDLC/Blueprints/BP_Test
[2020.04.23-12.09.40] Log: Package Path : /TestDLC/Blueprints
[2020.04.23-12.09.40] Log: Asset Name : BP_Test
[2020.04.23-12.09.40] Log: Asset Class : Blueprint
[2020.04.23-12.09.40] Log: Object Path : /TestDLC/Maps/DLCMap.DLCMap
[2020.04.23-12.09.40] Log: Package Name : /TestDLC/Maps/DLCMap
[2020.04.23-12.09.40] Log: Package Path : /TestDLC/Maps
[2020.04.23-12.09.40] Log: Asset Name : DLCMap
[2020.04.23-12.09.40] Log: Asset Class : World
[2020.04.23-12.09.40] Log: Object Path : /TestDLC/Materials/MI_Table.MI_Table
[2020.04.23-12.09.40] Log: Package Name : /TestDLC/Materials/MI_Table
[2020.04.23-12.09.40] Log: Package Path : /TestDLC/Materials
[2020.04.23-12.09.40] Log: Asset Name : MI_Table
[2020.04.23-12.09.40] Log: Asset Class : MaterialInstanceConstant
[2020.04.23-12.09.40] Log: Object Path : /TestDLC/Materials/M_Chair.M_Chair
[2020.04.23-12.09.40] Log: Package Name : /TestDLC/Materials/M_Chair
[2020.04.23-12.09.40] Log: Package Path : /TestDLC/Materials
[2020.04.23-12.09.40] Log: Asset Name : M_Chair
[2020.04.23-12.09.40] Log: Asset Class : Material
[2020.04.23-12.09.40] Log: Object Path : /TestDLC/Materials/M_Color.M_Color
[2020.04.23-12.09.40] Log: Package Name : /TestDLC/Materials/M_Color
[2020.04.23-12.09.40] Log: Package Path : /TestDLC/Materials
[2020.04.23-12.09.40] Log: Asset Name : M_Color
[2020.04.23-12.09.40] Log: Asset Class : Material
[2020.04.23-12.09.40] Log: Object Path : /TestDLC/Meshes/SM_Chair.SM_Chair
[2020.04.23-12.09.40] Log: Package Name : /TestDLC/Meshes/SM_Chair
[2020.04.23-12.09.40] Log: Package Path : /TestDLC/Meshes
[2020.04.23-12.09.40] Log: Asset Name : SM_Chair
[2020.04.23-12.09.40] Log: Asset Class : StaticMesh
[2020.04.23-12.09.40] Log: Object Path : /TestDLC/Meshes/SM_Table.SM_Table
[2020.04.23-12.09.40] Log: Package Name : /TestDLC/Meshes/SM_Table
[2020.04.23-12.09.40] Log: Package Path : /TestDLC/Meshes
[2020.04.23-12.09.40] Log: Asset Name : SM_Table
[2020.04.23-12.09.40] Log: Asset Class : StaticMesh
[2020.04.23-12.09.40] Log: Object Path : /TestDLC/Textures/T_Chair_M.T_Chair_M
[2020.04.23-12.09.40] Log: Package Name : /TestDLC/Textures/T_Chair_M
[2020.04.23-12.09.40] Log: Package Path : /TestDLC/Textures
[2020.04.23-12.09.40] Log: Asset Name : T_Chair_M
[2020.04.23-12.09.40] Log: Asset Class : Texture2D
[2020.04.23-12.09.40] Log: Object Path : /TestDLC/Textures/T_Chair_N.T_Chair_N
[2020.04.23-12.09.40] Log: Package Name : /TestDLC/Textures/T_Chair_N
[2020.04.23-12.09.40] Log: Package Path : /TestDLC/Textures
[2020.04.23-12.09.40] Log: Asset Name : T_Chair_N
[2020.04.23-12.09.40] Log: Asset Class : Texture2D
[2020.04.23-12.09.40] Log: Object Path : /TestDLC/Textures/T_TableRound_M.T_TableRound_M
[2020.04.23-12.09.40] Log: Package Name : /TestDLC/Textures/T_TableRound_M
[2020.04.23-12.09.40] Log: Package Path : /TestDLC/Textures
[2020.04.23-12.09.40] Log: Asset Name : T_TableRound_M
[2020.04.23-12.09.40] Log: Asset Class : Texture2D
[2020.04.23-12.09.40] Log: Object Path : /TestDLC/Textures/T_TableRound_N.T_TableRound_N
[2020.04.23-12.09.40] Log: Package Name : /TestDLC/Textures/T_TableRound_N
[2020.04.23-12.09.40] Log: Package Path : /TestDLC/Textures
[2020.04.23-12.09.40] Log: Asset Name : T_TableRound_N
[2020.04.23-12.09.40] Log: Asset Class : Texture2D
[2020.04.23-12.09.40] Log: Loaded Object Name : Fire01
[2020.04.23-12.09.40] Log: Loaded Object Name : Fire01_Cue
[/SPOILER]

I tested both 4.22 and 4.23 test projects and they are fine. unfortunately our game is and will stay in 4.24…

[BlueMountainsIO]

@sivan
Sorry for the late reply. I’ve been getting this serialization error in 4.24 as well. Even without mounting any additional paks, so just with the default project pak that Unreal creates. Haven’t been able to trace this down either.
One of the reasons why I can’t upgrade my projects to 4.24, unfortunately this version is not very stable for me.
Haven’t seen this in the new 4.25 preview so far. So that might be an option for you once it comes out.

[sivan]

hi,
no problem, and thanks for the info, basically the worst possible news, but I thought there is something different in the current engine version…
the game was updated from 4.18 to 4.24 in Feb because of significant rendering improvements, but resulting in many bugs, still not totally finished, so after releasing the current update, I hardly can imagine another engine version update soon If you don’t mind I will ask you again about your experiences with 4.25 later…

what I should solve is to make the game moddable, if possible without sharing the full game project, as it is mainly in Bp, and this plugin would be ideal for me to read in pak files by the game, created with a different, small mod editor project.
currently I can solve mounting/using multiple pak files in the game project made with the mod editor project only in a hacky way that I want to avoid (the 2 projects must have similar names, and include the same plugin for creating and receiving pak files created as DLC-s, but reading in multiple pak files referencing to the same plugin could lead to asset registry problems I think, especially if modders use similar file names)

if anybody has any good news/experience with UE4.24 please do not keep it in secret

[Rareden]

@sivan

hi,

if anybody has any good news/experience with UE4.24 please do not keep it in secret

@sivan We have had good experience with this on 4.24 along with our mod community,
only issue ive noticed is with landscapes, @BlueMountainsIO for some reason when loading into a mounted pak map containing a landscape this funky thing happens with the landscape rendering, it seems to shift up a mile and be flat. It only happens on a few users, majority of people it works fine

[Dash-POWER]

@BlueMountainsIO

Hello,

I bought the plugin but can’t make it working. I followed the documentation and tried example project (4.24). Everything built successfuly but when I run the shipped project, it does nothing. I guess there should appear some objects in the scene from the external .pak. I changed the paths in BP_Test as well. Any idea what might be wrong?

Thank you.

[BlueMountainsIO]

@Dash-POWER
Make sure to enable the runtime log and print out the results of every function to trace it down. You can find more info in the documentation about that.
Hope it helps.

[sivan]

@Dash-POWER

unfortunately the UE4.24 example project does not work, but fortunately only because of the TestDLC.pak file.
If you create a new project, add a plugin, cook the plugin (maybe a preliminary game built is also required), and rename and copy the resulting pak file in the place of TestDLC.pak, it works, but in the pak handling BP of the example project all the strings should be properly set according to the names used in your other project and its plugin, thus if you use similar names, nothing to modify.
and make sure the path of the pak file and the log are set properly, the log file is your best friend

[elmauro81]

Hi, i got this plugin and i am trying to make it work. Does anybody know if i can have a project with an App that i can use as a base app, then create a level with its own content in a different project and package as a DLC and then add as DLC (PAK) in the base project, so i can use the base app and create the levels with different projects. Thanks!