Primer: Loading Content at Runtime
Article written by Sebastian T.
This document is intended to give an overview over the various aspects of loading content/assets at runtime. It is specifically focusing on cooked content and native Unreal assets.
This does not cover how to import other types of assets, like USD scenes or CAD data through the datasmith runtime feature.
Loading Pak Files
Automated Loading
The engine will mount available pak files on startup from the following locations:
- /Content/Paks/
- Engine/Content/Paks/
- Saved/Content/Paks/
Pak files created through the regular packaging process will contain a manifest file called AssetResgistry.bin which informs the AssetRegistry about the assets in the pak file, precluding the need to manually scan the contents.
Manual Loading
Pak files can be loaded on demand at runtime through the C++ API.
There is two options:
- Broadcasting the FCoreDelegates::MountPak delegate
This is a high level API that does not require a pointer to the FPlatformFilePak implementation. Instead the implementation listens to broadcasts of this delegate and initiates a mount call in response. - Calling FPlatformFilePak.Mount directly.
This is the direct pak file PlatformFile API to initiate mounting of a pak.
Console Commands
Note: Console commands are not available in shipping builds.
See FPakExec in IPlatformFilePak.cpp for implementation.
- Mount []
- Mounts a pak file, optionally takes a mount point as second parameter
- Unmount
Unmount a pak file. - PakList
Lists currently mounted pak files.
File Priorities
Paks are internally stored in a list sorted based on priority which is determined first by priority value of the pak file, then on mount order (newer first). File requests will return the matching file from the pak with the highest sort order.
Depending on the pak file locations and name additional priority can be applied to pak files. For example, Pak files designated as a patch will get 1000 added to their priority when loaded at startup. This ensures patched files receive a higher sort order, in effect overriding the same files from other paks. Patch paks can be identified by the “_P” suffix in their file name.
Manual mount requests accept a priority parameter in addition to the implicit priority given based on the pak file locations and name. See FPakPlatformFile::GetPakOrderFromPakFilePath and FPakPlatformFile::Mount.
Mount Points in Pak Files:
All files stored in a pak file use paths relative to the binary of the application being executed. This means that no virtual paths (like /Game/) exist at this level yet.
Each pak file has a mount point that specifies where all files it contains were originally stored e.g. in the content folder of the project.
One can think of it as extracting the files inside the pak file into a specific directory in a packaged game folder.
The mount point will also be shown in the log, the relevant messages look like this:
LogShaderLibrary: Display: ShaderCodeLibraryPakFileMountedCallback: PakFile '../../../UE_4_27/Content/Paks/pakchunk0-WindowsNoEditor.pak' (chunk index 0, root '../../../') mounted
LogShaderLibrary: Display: ShaderCodeLibraryPakFileMountedCallback: pending pak file info (ChunkID:0 Root:../../../ File:../../../UE_4_27/Content/Paks/pakchunk0-WindowsNoEditor.pak)
Here is an example from a pak file (the output is from UnrealPak.exe -List):
LogPakFile: Display: Mount point ../../../
...
LogPakFile: Display: "Engine/Plugins/Tests/EditorTests/EditorTests.uplugin" offset: 28508892, size: 496 bytes, sha1: 87D598071C6B751B93866E831E91ECE50798A088, compression: None.
LogPakFile: Display: "Engine/Plugins/Tests/ScreenshotTools/ScreenshotTools.uplugin" offset: 28509441, size: 504 bytes, sha1: 55CDCB06BED437F1EB2A2554A841997D24B5FC25, compression: None.
LogPakFile: Display: "Engine/Plugins/VirtualProduction/Takes/Takes.uplugin" offset: 28510208, size: 1195 bytes, sha1: 713B557C2204E111153B9BFC917CAEC24BAA18B0, compression: None.
LogPakFile: Display: "UE_4_27/AssetRegistry.bin" offset: 28512256, size: 25530 bytes, sha1: 56E89C57180C4E02FB569451408B6705871F1C90, compression: Zlib.
LogPakFile: Display: "UE_4_27/Config/DefaultEngine.ini" offset: 28537859, size: 304 bytes, sha1: 39E94404601CF415DFEAC8F47CC87AEE4C3C7898, compression: None.
LogPakFile: Display: "UE_4_27/Config/DefaultGame.ini" offset: 28538880, size: 1170 bytes, sha1: B3708C02A48FEE414C5360A624D217D51FDCFF88, compression: None.
LogPakFile: Display: "UE_4_27/Content/BPTestLevel.uexp" offset: 28540928, size: 242419 bytes, sha1: 8C13800A24A77B7B7169812C45FEEB8D50333E6F, compression: Oodle.
LogPakFile: Display: "UE_4_27/Content/BPTestLevel.umap" offset: 28784640, size: 3684 bytes, sha1: 62996C8E4FD44390D36181C3859D089F91B5748B, compression: Oodle.
This file contains engine content, game content, config files, engine plugin files and more, each of those are mapped into the file structure of the packaged application. Mount Points from the Unreal File System are then used to sort these folders into in-game references like /Game/…
Mount Points / Root Paths in the UFS (Unreal File System):
Unreal uses a virtual file system for any asset or object references, called the Unreal File System. Asset References do not specify the absolute path on the disk, but instead refer to Root Paths which act like virtual folders that certain assets belong to.
The mount points map from a certain path on the disk to a virtual path that’s used for accessing all assets and references in game.
The paths used for creating a mount point are always relative to the actual game executable, which is usually located in “/Binaries//” (for a packaged project).
So the mount point “…/…/…/” should usually be your project’s folder.
Users should be aware that these assume the folder structure of a packaged project, so the project content will be in /Content/ instead of Content/.
There are several default mount points that the engine will setup:
- /Game/ which is mapped to the project’s content folder (…/…/…//Content/)
- /Engine/ which is mapped to the engine’s content folder (…/…/…/Engine/Content/)
Some temporary/read-only ones:
- /Script/ for classes
- /Temp/
- /Config/
One mount path per Plugin:
- /PluginName/ for each Plugin content folder, e.g. Plugins/PluginName/Content/
Plugin Loading Behavior:
At engine startup the FPluginManager will attempt to discover and load all enabled plugins.
This is done by searching for any available “.upluginmanifest” files, and (if none are found) manually searching for “.uplugin” files.
When a plugin is loaded it’s content directory will be added as a Root Path /PluginName/.
Note: Plugin content Paks may contain a serialized AssetRegistry, but it will only be used if the plugin was loaded at game startup. it will not work when the pak file for a plugin is downloaded and mounted at runtime, in this case the Root Path for the new plugin has to be added manually by the developer.
DLCs
When building DLCs the regular workflow is to package a named/versioned release of the game (for example “1.0”) and then add additional plugins to the project later. New plugins can be packaged as a DLC plugin and added to an already packaged game to make it available to users.
If the engine discovers the plugin as a DLC plugin it should follow the regular Plugin mounting logic, whereas when you only mount the pak file from your code you may need to specify the mount point manually.
Note: On PC an additional flag for the package command may be required when building a DLC so that it is properly recognised by the engine: “-DLCPakPluginFile”.
This flag will include a file called PluginName.upluginmanifest in the pak file and allow the engine to detect that a DLC Pak file contains content that belongs to a previously unknown plugin. As with the AssetRegistry.bin this will require the pak file to be available at engine startup.
AssetRegistry Behavior:
The AssetRegistry only knows about the assets that were present at cook time. It loads this cached information during startup from the serialized version of the AssetRegistry (AssetRegistry.bin) which is stored inside pak files.
Unreal will load a number of AssetRegistry.bin files during startup, for example the AssetRegistry for each plugin is loaded on startup, if it is present.
Additionally the AssetRegistry listens to the FPackageName::OnContentPathMounted delegate, so when a new mount point is mounted the Asset Registry will start to asynchronously search the new directories for any assets and add them to the internal database.
This will be asynchronous and the asset might not be available immediately.
IAssetRegistry::ScanPathsSynchronous can be used to force an immediate update of the asset cache and make new assets available through the asset registry in the same frame.
Limitations on Loading External Assets at Runtime
There are limitations in the low level file APIs (IPlatformFile) in Unreal that prevent loading of loose .uasset/.umap and other Unreal asset files from a packaged game.
FPlatformFilePak:
This is the default filesystem accessor that is used in a packaged game. It will load any files from the mounted pak files by default and fall back to a regular file system access if a file cannot be found. There are additional security checks for accessing files that are outside of any pak file.
FPakPlatformFile::ExcludedNonPakExtensions specifies a list of forbidden file names outside of pak files. If the user is compiling the engine from source this can be overridden by setting the preprocessor define EXCLUDE_NONPAK_UE_EXTENSIONS to 0.
This can be done in the *.Target.cs file of a game like this:
GlobalDefinitions.Add(“EXCLUDE_NONPAK_UE_EXTENSIONS=0”);
Default forbidden file extensions in 4.27 when using pak files:
- uasset
- umap
- ubulk
- uexp
- uptnl
- ushaderbytecode
FSandboxPlatformFile:
This will be used when starting a game with cooked content, but without a pak file. This usually happens when starting a non-editor build from Visual Studio without packaging the game first and requires cooked content to be available (cook content for windows has to be executed at least once). The default folder for the cooked content is Saved/Cooked/WindowsNoEditor.
Sandbox mode requires the “-sandbox=” argument with the full path to the cooked content and is only available for client and game builds on desktop platform.s
Default forbidden file extensions in 4.27 when starting with cooked content but without .pak files (sandbox mode):
- uasset
- umap
The recommended solution for loading additional game content at runtime is to use pak files as described above.
Existing Solutions for Dynamic Content Loading
There are multiple implementations in the engine that already provide logic to acquire and load content packages at runtime. It’s out of the scope of this document to cover them in detail, but they are mentioned here for reference and should be preferred over a full custom solution if possible.
The ChunkDownloader provides a solution for creating individual content pak files out by using the built-in chunking system and integrates logic to download a list of available pak files from a server, download a user-defined subset of these files and mount them on demand.
It can serve as a good example for building custom logic or it can be used as is for many simpler use-cases.
Additionally many platforms provide their own implementations of a chunked install system, see IPlatformChunkInstall in GenericPlatformChunkInstall.h for an overview of the API.
Get more answers on the Knowledge Base!