Hey team,
We’ve got a pretty large Android improvement to share.
To set some background, we’re building for mobile, and utilizing Chaos Physics in async mode (To make use of fixed timestep physics) - additionally, we’re utilizing the re-simulation replication mode, because it suits the needs of our title.
Async physics in Unreal Engine doesn’t run on a dedicated thread, but rather runs in the Taskgraph task pool threads.
We’ve seen some really inconsistent performance on Android, we narrowed it down to poor scheduling decisions within the OS, which can be resolved by integrating with the PerformanceHintManager (PerformanceHintManager | API reference | Android Developers).
The problem presents to different extremes on different devices, with Pixel devices being the worst case for us
In our testing we’ve been able to improve our frame rate stability on Android immensely by taking Googles ADPF plugin for the engine (ADPF Unreal Engine plugin | Android Developers), and modifying it to include the task graph threads in the game thread hint session.
That change drastically improved our frame rate variability by informing the OS of the performance critical nature of those threads.
Our implementation is quite rudimentary, we exposed a getter for the worker thread IDs from LowLevelTasks.FScheduler and includes those in the hint session along with the game thread.
TArray<uint32> FScheduler::GetWorkerThreadIds() const
{
TArray<uint32> WorkerThreadIds;
for (FThread* WorkerThread : WorkerThreads)
{
WorkerThreadIds.Add(WorkerThread->GetThreadId());
}
return WorkerThreadIds;
}
You can see the results in the attachment “Performance-comparison.png”
Some problems with this implementation include:
- The scheduler has logic for restarting threads, and we’re not really sure yet of the lifetime expectations of those threads. We just grab them on app startup, so if they’re ever re-created, the newly created thread pool won’t be part of our game thread hint session.
- This also includes both the foreground and background threads as performance critical threads, but background threads probably don’t need to be run on high speed cores (For better thermal performance) - however TaskGraph will schedule foreground work on background threads, so in the cases that happens, we actually do need the OS to run them on faster cores
- It would probably be best on Android to have more explicit foreground and background work scheduling so that frame critical work doesn’t occur on background threads which aren’t included in the hint session, or perhaps the threads could dynamically be added to the hint session by the scheduler. Not really sure on the ideal implementation.
Additionally, when using re-simulation replication, we hit a frame rate spiral of doom on Android quite frequently because of the lowered core speeds.
We’ve been able to fix this locally by running our game side code on its own thread(s), with their own performance hint session, and decreasing the time requirement for those threads to complete in response to a resimulation beginning, this has resolved that spiral of doom, but does still incur a decent frame rate drop initially due to the time it takes for the OS to either ramp clock speeds, or change its scheduling behavior to accomodate the performance hint session adjustment
So, big wall of text out of the way, I’d very much like to see the scheduler, and fixed threads (game / render / RHI) integrate the performance hint manager “out of the box” in engine.
It would also be great if async physics could be accounted for in the implementation - while we have improved these problems for our title, it would be nice if these improvements could be made to the engine (So that everybody can share in better mobile performance from the engine, and so that we’re not maintaining these divergences forever)
Thanks!
[Attachment Removed]