Platform Extensions (and how they may impact you)
Background
Unreal has a rich history of multi-platform support. We have had various methods of separating out the platform code from the rest of the engine but even our latest method of platform separation is not “trivial” to keep platform code nicely segregated:
- Perforce permissions are incredibly complex
- There are references (not API calls) to every platform hardcoded in the engine[/li][li]Adding a new platform touches many parts of the engine
- Protecting an unreleased platform from disclosure is complicated
To this end, we are introducing Platform Extensions. A Platform Extension is similar to a plugin that can be “dropped in” next to the Engine to create a new platform. This means that we can avoid platform references in the Engine code, Epic samples, project files, shaders… you get the idea.
Most existing platforms will not be be converted to a Platform Extension. In the future, we plan to have a Stadia Platform Extension, and we are considering possibly converting Xbox and PS4 as well. Win32/64, iOS, Linux etc will remain integrated as they are today, albeit with some notable changes (below).
Who is affected?
In order to enable Platform files to be discoverable across multiple locations, big changes have been made to UBT. Implementing Platform Extensions has also required changes in general engine code. The following is a list of developers who are very likely to be impacted:
- Developers who modify Engine code
- Platform developers (any platform)
- Developers who author/modify their project’s Build.cs and Target.cs files
- Developers modifying ini hierarchy
- Developers using the platform tables in PlatformInfo.cpp
- Rendering programmers
When does this change take place?
These changes are checked into //UE4/Dev-Build/ and have merged to //UE4/Main/ main as of CL 6631504.
The first release branch these changes are expected to land in will be 4.23.
Note:
This article covers fundamental changes to how platform files are handled throughout UE4’s codebase. Occurrences of [Platform] herein should be treated as an appropriate platform name (ie ‘Android’, ‘PS4’ or 'Windows’). For example, the Platforms/[Platform]/ folder mentioned below would be Platforms/PS4/ for the PS4 platform.
New directory structure
The most important part of the change is how the directories are laid out. The platform extensions live in [UE4Root]/Platforms/[Platform]/… This makes provisioning Perforce permissions much easier, keeps all restricted platform code in one place and generally is much more tidy.
A project can also have [ProjectRoot]/Platforms/… , but it is optional. You may leave platform specific code and other files in their original location, but Epic projects will have their files in the new structure. (If you are distributing your project, you may want to refactor the files to keep NDAd platforms separated).
The old directory structure, whereby platform files are distributed all throughout the codebase according to the feature they implement, will remain mostly intact, except for the changes mentioned below.
Sample Platform Extension
For a very basic example of a Platform Extension, we have created a sample platform named ‘XXX’ located in the /Platforms/XXX/ directory which shows how this new approach works in practice.
Rules for Platform Extensions
Given these changes, the following assertions apply to all Platform Extensions:
All files for the Platform must live inside [Engine|Project]/Platforms/[Platform]/ , including Developer modules, .Build.cs subclasses, Plugins, .inis, etc.
- PLATFORM_[Platform] is not defined for Engine modules!
- A module must set bAllowConfidentialPlatformDefines to true in order to have it set
- Any ShaderPlatforms are defined in DataDrivenPlatformInfo.ini
- PlatformInfo objects are defined in DataDrivenPlatformInfo.ini
- The UnrealTargetPlatform for the platform is not referenced from outside the platform-specific .cs files
Workflow Impact
UnrealBuildTool
Impacted Audience: Anyone working in Build.cs and Target.cs files
Description: UnrealTargetPlatform was previously an enum with all platforms listed. To remove the hardcoded platform names, UnrealTargetPlatform has been changed to a class with static members named the same as the enum values. Additionally, it is a partial class, so a Platform Extension can extend the class to add a member with the platform’s name.
In general, it should act the same as enum (you can compare them, assign them, etc). However, since they are static members, they are not a constant value, so they cannot be used in a switch statement. All existing uses in the engine have been converted to if statements instead. In addition, the SupportedPlatforms attribute on a Target subclass now uses strings that match the platform name instead of the UnrealTargetPlatform.
This is likely to be the single biggest impact on you as a developer, unless you have also made engine/platform code changes.
HAL Header Filename Standardization
Impacted Audience: All Platform programmers (additional notes for Windows, TVOS, PS4, XboxOne, and Android developers)
Description: Platform references have been removed from the Hardware Abstraction Layer (HAL) “uber” headers, such as PlatformTime.h (used below as an example). Platform-specific versions of these header files are now referenced in the following manner:
#include COMPILED_PLATFORM_HEADER(PlatformTime.h)
The COMPILED_PLATFORM_HEADER macro expands to add the current being compiled as a prefix. As such, all platform header file names must be standardized to the form [Platform]PlatformTime.h
PS4, XboxOne, and Android:
Historically these platforms did not have the word “Platform” in their header file names (e.g. PS4Time.h), which was in contrast with other platforms. These headers and source files have been renamed in order to make them consistent with other platforms (e.g. PS4Time.h is now PS4PlatformTime.h).
Platforms which are not converted to Platform Extensions will still have these files renamed, so developers that have made changes to affected platform files will need to make sure to move their changes over.
Win32, Win64 and TVOS:
Since Win32 and Win64 each share common headers, and since TVOS uses the IOS headers, we have added a mechanism to override the platform name that results when expanding the COMPILED_PLATFORM_HEADER macro. The override is declared in UEBuild[Platform].cs in the SetupEnvironment method by adding a new CompileEnvironment definition with:
CompileEnvironment.Definitions.Add("OVERRIDE_PLATFORM_HEADER_NAME=[SharedPlatform]”);
Examples:
- Win32 and Win64 override to “Windows”, so they would share “WindowsPlatformTime.h”
- TVOS overrides to “IOS”, so it would use “IOSPlatformTime.h”
Log categories for Platforms
Impacted Audience: Anyone writing engine code
Description: Platform log statements like UE_LOG(LogWindows, …) have used hardcoded Log categories, which were globally defined for all platforms. Now, the category is generated based on a define that is only set on that platform. This means that if you use UE_LOG(LogWindows, …) on any other platform than Windows, it will no longer compile. This should be an easy fix
Ini File Specification
Impacted Audience: Anyone who has added or will add new .ini files into the ini hierarchy.
Description: We have simplified the ability to search for .ini files in deep directory structures. The current implementation used a rigid list of files, which has now been replaced with a more programmatic solution we call “Ini Layers”. This concept breaks up the ini stack into discrete layers which can each be expanded according to various factors. Consider the following ini stack, which we can describe as having 5 layers:
- Engine/BaseEngine.ini
- Engine/Base[Platform]Engine.ini
- Project/DefaultEngine.ini
- Engine/[Platform]Engine.ini
- Project/[Platform]Engine.ini
The Ini Layer feature enables us to expand each layer of the ini stack into multiple ini files according to the permutations appropriate for that layer.
For instance, Epic uses NotForLicensees and NoRedist as directories to segregate certain confidential/secret settings into directories with reduced distribution, which increases the permutations for each of these layers.
Additionally, this new feature expands upon our existing support for Ini Platform Parents, which now allow for a chain of parents, rather than just a single platform parent.
For example, suppose you have three platforms, A, B and C which are related in a chain; Grandparent A, Parent B and Child C. Ini settings for platform A (the grandparent) should be inherited by both A and B. Likewise, platform B settings should be inherited by platform C.
Previously, we did not have a way to specify platform A, and interlacing the appropriate ini files to cause C to inherit B’s settings was a challenge. Using Ini Layers, this full chain is now possible and would result in an ini stack for platform C as follows:
- Engine/BaseEngine.ini
- Engine/Base[…]Engine.ini
- Engine/Base[A]Engine.ini
- Engine/Base[B]Engine.ini
- Engine/Base[C]Engine.ini
- Project/DefaultEngine.ini
- Engine/[…]Engine.ini
- Engine/[A]Engine.ini
- Engine/[B]Engine.ini
- Engine/[C]Engine.ini
- Project/[…]Engine.ini
- Project/[A]Engine.ini
- Project/[B]Engine.ini
- Project/[C]Engine.ini
The specification of these Layers (GConfigLayers) and Expansions (GConfigLayerExpansions) is in ConfigCacheIni.cpp. See also EnumerateConfigFileLocations which defined in ConfigHierarchy.cs.
Two layers from the GConfigLayers definition are shown below. These definitions use bracketed tokens which are expanded to valid permutations when generating the ini file names.
// Project/Default*.ini
{ TEXT("ProjectDefault"), TEXT("{PROJECT}{ED}{EF}Default{TYPE}.ini"), TEXT(""), Flag_AllowCommandLineOverride | Flag_GenerateCacheKey },
// Project/Platform/Platform*.ini
{ TEXT("ProjectPlatform"), TEXT("{PROJECT}{ED}{PLATFORM}/{EF}{PLATFORM}{TYPE}.ini"), TEXT("{EXTPROJECT}/{ED}{EF}{PLATFORM}{TYPE}.ini") },
Valid tokens are as follows:
- {PROJECT}
- The project directory
- Has no permutations
- Token ends in a ‘/’
- {ENGINE}
- The engine directory
- Has no permutations
- Token ends in a ‘/’
- {TYPE}
- The type of ini file it is
- Expands for each ini type; Game, Input, Engine, Editor, etc
- {ED}
- The DirectoryPrefixs defined in GConfigLayerExpansions
- NotForLicensees, NoRedist, etc
- Expands for each DirectoryPrefix in GConfigLayerExpansions
- Token ends in a ‘/’
- The DirectoryPrefixs defined in GConfigLayerExpansions
- {EF}
- The FilePrefixs defined in GConfigLayerExpansions
- DedicatedServer, etc
- Expands for each FilePrefix specified in GConfigLayerExpansions
- The FilePrefixs defined in GConfigLayerExpansions
- {PLATFORM}
- The current platform and any parent platform(s) in the IniParentChain
- Expands once for each platform in the chain
Note that your existing .ini files will still work. This only changes the way the hierarchy is specified, enabling more files to be easily added, and adds the PlatformExtension location to the search.
PlatformInfo structure
Impacted Audience: Anyone modifying the large table in PlatformInfo.cpp
Description: We currently specify information about every platform in PlatformInfo.cpp. This naturally needed to change to remove platform information from the engine. The settings are now discovered by looking for DataDrivenPlatformInfo.ini files in each platform’s Engine/Config/ directory.
Currently, the DataDrivenPlatformInfo.ini files are not project-extendable, as they are meant to simply describe a platform. However, in the future this may change.
ShaderPlatforms
Impacted Audience: Rendering programmers
Description: A ShaderPlatform is used to define characteristics of rendering capabilities of a rendering API on a platform. The information about these ShaderPlatforms have always been hard-coded in C++, which need to be removed.
A ShaderPlatform can now be specified in a DataDrivenPlatformInfo.ini via the FDataDrivenShaderPlatformInfo class. This class will have more settings added to it as the rendering code adds more capabilities to check.
Currently, the existing ShaderPlatforms have not been converted to .ini-driven, so only about half of the functions named things like RHISupportsXxxxxx make use of the new system. This is a work in progress.
