Pak File Loader Plugin

[BlueMountainsIO]

Support thread for the Pak Loader plugin.

Allows you to mount .pak files through Blueprints and load assets.

Features

• Mount/Unmount .pak files.
• Register mount points to support references between pak assets.
• Download (.pak) files via HTTP.
• Load objects (all kind of assets).
• Load classes (Blueprints).
• Utility functions like GetMountedPakFilenames, GetFilesInPakDirectory, SHA1SUM, IsValidPakFile, etc.

Marketplace
https://www.unrealengine.com/marketp...-loader-plugin

Documentation (Read before buy)
https://drive.google.com/file/d/1DeS...AHs89rDUM/view

Example Project (Requires the plugin; read documenation first)
https://drive.google.com/file/d/1W6j...jiRkOrLIE/view

[MagicBots]

MagicBots
Sorry for the late reply. I can look into making the logging available to packaged development builds as well.

Glad that you have it got working. Good question to what happens when there you mount paks that have files that already exist. I think that the last mounted pak would win. Maybe a rescan of the ModPlugin/ path would work.

That's how I got them registering was a rescan. For some reason rescanning with the root path throws a fatal error saying the path doesn't map to a known root . I tried making sure it's mapped just before the scan.. and i assume it's mapped because it's working, but for some reason scanning the path always threw that error for me no matter what i tried. maybe it has to do with the fact that unreal has no way to know about the plugin root or something because it's packaged separately in the case of a mod.

I'd rather scan by root path as it seems more efficient.. Would you know how to solve the unknown root for plugin root paths packaged outside of where the main game is packaged problem? Not sure how to properly register root in this case. I guess maybe that ModSkeleton setup is doing it but I couldn't get it working.

[BlueMountainsIO]

MagicBots
Sorry for the late reply. I can look into making the logging available to packaged development builds as well.

Glad that you have it got working. Good question to what happens when there you mount paks that have files that already exist. I think that the last mounted pak would win. Maybe a rescan of the ModPlugin/ path would work.

[MagicBots]

BlueMountainsIO I was wondering if it's a necessary requirement to package for shipping for this to work, and if so why? Past experiments I'd done with paks, had them working fine when packaging for development. I know there is the loghelper you've provided for logging when packaging in shipping mode, but for other reasons i was wanting to have it work packaged for development as well. I've tested PakLoader with development mode and it all seems to work the same for me as i can tell so wanted to check.

Also mounting and registering are both working for me at Engine level ../../../ as was suggested, however I'm mounting multiple paks into the same mount root /ModPlugin/, and the AssetRegistry.bin that I'm reading won't update registry since the AssetRegistry.bin file always has the same path. Is there a way to have the AssetRegistry.bin append to the existing registry at the mountroot?

Essentially I have a whole bunch of small paks with a few files in them that I want to mount and register beneath the same root each time.. But the AssetRegistry.bin won't register because there was already an existing one at the path for that mount root ( it seems ). Not sure how to solve this. If I don't read in AssetRegistry.bin then I have to spawn them with your getpakfilestaticmesh in order for the registry to know about them. What I'm seeing currently is the first AssetRegisty.bin I read updates the registry and the rest don't update so i'm stuck with the registry only knowing about the first one..

My asset AssetRegistry.bin in all of my small paks is always at:
Pod/Plugins/ModPlugin/AssetRegistry.bin

Is there a way I can manually add the object paths mounted from the pak to the registry object by object instead of serializing the AssetRegistry.bin?

EDIT:

Nevermind I got it registering!

[Rareden]

MagicBots
Need a extra / at the end there,
The code line is simply this

  
 // Add the base path to the file list array so the mount point is correct and assets can be accessed through /Game/
  FileList.Add(TEXT(".. / .. / .. /"));

It dosnt matter where in the response file list it is, just that it has it in there.

[MagicBots]

Rareden, what does the line need to look like exactly? I put "../../.." as its own line in the top of the response file and it zipped my whole project into the pak =)

[Rareden]

Snip

Ty good sir, its working now.
I had to Append the" ../../../" into the response file array as a individual index to get the unrealPak to have the mount point as "../../../" in case anyone else runs into the problem too

[MagicBots]

Rareden

@MagicBots
I would assemble the pak so it looks like this.

Mount point: ../../../

"Pod/Plugins/ModPlugin/Content/Cube.uasset"
"Pod/Plugins/ModPlugin/Content/M_Cube.uasset"
etc.

Then your RootPath is "/ModPlugin/" and your ContentPath is "../../../Pod/Plugins/ModPlugin/Content"

ok, trying this. I noticed that i'm getting a .pak generated for the mod packaging approach that looks like this:

Pod\Plugins\ModPlugin\Content\Paks\WindowsNoEditor\ModPluginPod-WindowsNoEditor.pak" -List
LogPakFile: Display: Using command line for crypto configuration
LogPakFile: Display: Added 0 entries to add to pak file.
LogPakFile: Display: Mount point ../../../Pod/Plugins/ModPlugin/
LogPakFile: Display: "AssetRegistry.bin" offset: 0, size: 44 bytes, sha1: CC66D83CB38DE2F3A30822A68EB38A3865471816, compression: None.
LogPakFile: Display: "Metadata/DevelopmentAssetRegistry.bin" offset: 2048, size: 143066 bytes, sha1: A9820C94D2CDF676DD30655C61BBBC57E5E560DF, compression: None.
LogPakFile: Display: 2 files (143110 bytes), (0 filtered bytes).
LogPakFile: Display: Unreal pak executed in 0.003891 seconds

If i want to read the AssetRegistery with the .pak I create for my cube, do i add it with the same approach as content
(same mount points etc) or is there a different procedure?

For some reason I cannot get my packaging approach
for my mod plugin to include the content both the asset registry and the content.. So i'm having to manually pak the
content. Not sure why.

I suppose I can also manually pak the registry in with the content but maybe it's better to keep
them separate if the registry needs a different rootPath and contentPath generally..?

[BlueMountainsIO]

Rareden
That's a really strange pak layout. If you want the assets to be available in /Game then this will work (Tested and it works):

Pak File layout:



"ModProject": Name of the Unreal project. All content is from /Game
Mount point must be ../../../

Mount with the Pak Loader plugin:
MountPakFile("D:\MyGamePak.pak", "")<- Leave the mount point parmeter empty (mount point from the pak will be used which is ../../../)
RegisterMountPoint("/Game/", "../../../ModProject/Content/") <- RootPath and ContentPath.
(Optional) LoadPakAssetRegistryFile("../../../ModProject/AssetRegistry.bin")

Now you can load assets like this: GetPakFileStaticMesh("/Game/Tents/Meshes/SM_Tent")


@MagicBots
I would assemble the pak so it looks like this.

Mount point: ../../../

"Pod/Plugins/ModPlugin/Content/Cube.uasset"
"Pod/Plugins/ModPlugin/Content/M_Cube.uasset"
etc.

Then your RootPath is "/ModPlugin/" and your ContentPath is "../../../Pod/Plugins/ModPlugin/Content"

[MagicBots]

I need to fork/update repo the correct way still, but here is ModSkeleton for 4.22:
https://drive.google.com/open?id=1Fx...Pwv5oOP0cQx7t2

If you setup the 2 package profiles correctly it should work, I changed the .bat file because it did not have the correct filename to copy the .pak properly.
Cooking should not fail, only gives warnings because it thinks it needs those references but it does not. You should also be able to access the AssetRegistry from within the modpak at runtime with the PakLoader plugin...

There must be something i'm missing because the cook fails for me even with this version. The only way that ModPluginA build (cook/package) works is if I keep the plugin enabled when i build the first main build, which i think defeats the purpose of having the main build built without knowledge of the plugin. I've went over project launcher settings several times so it must be something else. As it's unrelated to PakLoader, I'll send you a p.m. to what I'm seeing. Definitely helpful for my understanding anyway even with them not working with disconnected builds. so thanks.

[Cytokinetik]

I need to fork/update repo the correct way still, but here is ModSkeleton for 4.22:
https://drive.google.com/open?id=1Fx...Pwv5oOP0cQx7t2

If you setup the 2 package profiles correctly it should work, I changed the .bat file because it did not have the correct filename to copy the .pak properly.
Cooking should not fail, only gives warnings because it thinks it needs those references but it does not. You should also be able to access the AssetRegistry from within the modpak at runtime with the PakLoader plugin...

[MagicBots]

This may be of some help about learning the .pak system and asset registry:
https://github.com/nakata0705/ModSkeleton

This was forked and updated from the original, it is a modding skeleton method. I think the assetregistry.bin for addon paks is also located and cooked inside of the .pak as well. I am not sure if that could cause a conflict. Using this .pak loader plugin along with the modskeleton method may help the process. The modskeleton allows adding .paks/mods without needing the original project at all...

Regardless it is a good learning tool along with the .pakloader plugin,

@Cytokinetik. This is great example and just what I needed for knowing what I was missing in terms of building a mod, however, it isn't a working example for me on 4.22 unfortunately. Only the 'cooked' directory gets made for the content plugin during build, and not the 'StagedBuild' directory which is of course needed for this command that copies the pak file from the plugin 'StagedBuild' to the mod directory:

copy /Y .\Plugins\ModSkeletonExamplePluginA\Saved\StagedBuilds\WindowsNoEditor\ModSkeleton\Plugins\ModSkeletonExamplePluginA\Content\Paks\WindowsNoEditor\ModSkeletonExamplePluginA.pak .\Mods\ModSkeletonExamplePluginA.pak

I'm not sure how to fix it to make cooking not fail, and packaging to make the StageBuild. The cook command fails half way through, which they say is fine, but clearly is not. Would you know if there is a newer working example for 4.22?

[MagicBots]

Please show the output of the UnrealPak -List command. Also take a look at the documentaiton, the function LoadPakAssetRegistryFile is explained there. I would also suggest to try out the demo project.

By the way, from my testing, it's not necessary to load the AssetRegistry.bin file manually. Just use the loading functions from this plugin and it will work.

It is working to spawn my assets. They are downloading and mounting fine. The static meshes are only showing up in the asset registery only after being spawned once howerver, and BP's I can't register with assetRegistry yet.

Here's an output of my UnrealPak.exe -List for an example asset:

LogPakFile: Display: Using command line for crypto configuration
LogPakFile: Display: Added 0 entries to add to pak file.
LogPakFile: Display: Mount point ../../../Pod/Plugins/ModPlugin/Content/DLC/13034129-7933-4fb8-86a0-8fb876d20c72/
LogPakFile: Display: "Cube.uasset" offset: 0, size: 1228 bytes, sha1: AEE2E38AACAA4C47A5E9E0F04EE6C20235235A1F, compression: Zlib.
LogPakFile: Display: "Cube.uexp" offset: 1298, size: 2326 bytes, sha1: D5D6A206484825737425B2A2E88018959C2167AF, compression: Zlib.
LogPakFile: Display: "M_Cube.uasset" offset: 3694, size: 504 bytes, sha1: 19929E4F09BCF08ECBF8FDA1927BE1E1A4B29AB3, compression: Zlib.
LogPakFile: Display: "M_Cube.uexp" offset: 4268, size: 43501 bytes, sha1: F68F832E786E47CBB837FCEDC41066881636B344, compression: Zlib.
LogPakFile: Display: 4 files (47559 bytes), (0 filtered bytes).
LogPakFile: Display: Unreal pak executed in 0.005045 seconds

[Rareden]

Unfortunately, the images you uploaded do not display.

BlueMountainsIO Thats weird, Attached the images instead
Is the ../../../ before the mount point important?

[BlueMountainsIO]

I have also been trying to simply move in the packaged content plugin to the packaged game plugin location, with the cooked AssetRegistry.bin placed where BlueMountainsIO plugin is expecting it according to the example. No success with having the registry successfully appended via the LoadPakAssetRegistry function thus far ... Hoping BlueMountainsIO can advise as to how that was working with his example which seems to also have the AssetRegistry.bin file being expected at a plugin location. Everything else is working, just not the registry..

Please show the output of the UnrealPak -List command. Also take a look at the documentaiton, the function LoadPakAssetRegistryFile is explained there. I would also suggest to try out the demo project.

By the way, from my testing, it's not necessary to load the AssetRegistry.bin file manually. Just use the loading functions from this plugin and it will work.