GPU Benchmark (SynthBenchmark) gives different GPUPerfIndex values in 5.5

Development Rendering
, ,
1

Hello,

We are relying on the benchmark to automatically choose the default scalability of our game.

Since 5.5 (I can reproduce in 5.6 preview from Epic Games Launcher) the GPUIndex value from the benchmark seems unreliable. The computed values are much lower, breaking our current scalability thresholds, and seem more prone to variance. For instance, a NVIDIA GeForce RTX 3070 gives me about 595 in 5.4 and 475 in 5.5. The same is true for an AMD Radeon RX 6800 XT, going from 875 to 575 (but also randomly shown 450 in a package). The same computer is always used for these tests.

I searched for potential changes in 5.5 but the only one found is the new warmup rounds (https://github.com/EpicGames/UnrealEngine/commit/a6d35552a87bd0ae1485d6ee52f0eeca58e0487e), and reverting it in our custom engine doesn’t change the result. Maybe the new parallel rendering optimizations and/or changes around render queries are the cause.

I’m sharing a text file containing logs of SynthBenchmark from 5.4 and 5.5, the most notable difference here being the ‘FillOnly’ results.

Are you aware of this issue and potential changes causing it?

Thank you

2

Steps to Reproduce

  1. Open Unreal Engine 5.4
  2. Use SynthBenchmark command
  3. Note the GPUIndex result
  4. Open Unreal Engine 5.5 (or 5.6 preview)
  5. Use SynthBenchmark command
  6. The GPUIndex result is not the same
3

Hi Ulysse,

Yes, we have had some folks report issues with the synth benchmark recently. However, that feature has not received any improvements recently, due to developer time being spent on other parts of the engine. So, it could be that the new parallel rendering optimizations have affected the consistency of the benchmark. Do you get more consistent results when you run the benchmark with your game in single-threaded mode (launch with -onethread -rdgimmediate command line arguments)?

4

Ok, that is not good to hear. Can I ask you to please upload the logs of that run as well, so that I can investigate this further? Thanks!

5

Thanks for sending the logs. I have gone ahead and filed a ticket with the engineering team to further investigate this issue. You can view the public link for it here: Unreal Engine Issues and Bug Tracker (UE\-290402\). You should be able to track this progress in the future, as the team will investigate it. If you have any further questions, please let me know.

6

Hello,

Thank you for the answer.

Unfortunately, I tried the single-threaded mode and the results were the same than the broken ones (5.5 and 5.6 Preview).

7

Sure, here’s the full logs of a standalone game launched with “-onethread -rdgimmediate” as additional parameters. You’ll find multiple benchmarks at the end of it.