Blueprint vs C++ performance .

[vebski]

I think during 4.2 I did some tests for my collage presentation. I used flocking algorithm for AI. I had group of agents starting in a dispersed formation and on a button trigger they would start the flocking.
Difference was around 117fps (C++) vs 72fps (BP) during calculations (they were calculating every frame until position in flock was reached). Maybe its not 10x better, but I would say the difference in performance for calculating heavy things is still painful in BP. Your tests seem totally accurate thanks for doing them @! its nice to compare them with my results

[gameDNA]

Maybe on desktop difference is not so significant but on mobile my game made with BPs had ~20 FPS and now made with C++ has ~60 FPS so it’s about 3 times faster.

[Pionwave]

Anyone have compile time comparisons for different cpu’s?

My i7-4790k is taking 20-30seconds to compile my plugins when I make changes

Have a feeling my old i7-3930k would have been better to hold onto

[DennisSoemers]

@;582478

Conclusions:

1. ok package versions much faster, logical since no overhead from editor
2. inside the editor, yes blueprint is always slower than C++ (10-40%)
3. inside the editor, the more actors you have, the more differences you will see between BP and C++ (at 2400 actors: BP 37,5% slower than C++, at 1200 actors: 25,5%)
4. in packaged versions, the differences between BP and C++ versions are much smaller: 8% for 1200 actors with mesh. But at 1200 actors with mesh, the Draw Thread is influencing the fps.
5. to remove the impact of Draw Thread, I removed the mesh of those 1200 actors. Stat unit & the 200 fps count confirmed that only the Game Thread is driving the fps. When starting the simulation, the frame rates drops from 200+ to 160 fps for the C++ version and 132 fps for the BP version. We can consider this result as a 21% brute gain performance of using C++ over BP.

**Global Conclusions: ** (only concerns packaged version as editor doesn’t matter for final products)

• Yes, C++ is faster than BP
• BUT C++ is not at all 10 times faster than BP, even for pure computations. If anyone as a test case showing this 10 times faster I would be very interested to see that
• In my tests, the gain of using C++ for specific computations was roughly 20%. I don’t expect this gain to be more than 30%, even for stronger computations
• BP is surprisingly very fast, I’ve read somewhere the development team is working on translating BP in C++, perhaps is it already partially done ? Anyhow BP is just an interface to actually call the C++ codes of UE4 platform.
• I would advise to use C++ only when tremendous calculations are involved as the performance gain is not as important as one would expect / hope OR when you need more flexibility than Blueprint for your project architecture. Note BP is still very flexible.

In my opinion this is not a particularly clean benchmark and you cannot draw these conclusions from the results. The main reason why I’m saying this is that you’re not cleanly comparing some pure C++ to some pure Blueprint (BP). The moment you drop an Actor into a level, you start running a lot of C++ code for the implementation of the AActor class (and also APawn class since I believe you mentioned actually using Pawns?), and there can be additional overhead in all kinds of other systems that have been implemented in C++ (for instance the Garbage Collector of UE4, and all kinds of other global systems). So if you compare a basic AI implementation for that Actor in BP vs C++, you’re actually comparing “Blueprint-based AI + a whole bunch of C++ behind the scenes” to “C+±based AI + a whole bunch of C++ behind the scenes”. A simple way to at least try to control for this (I’m not saying it’s a 100% complete correction, but should at least be better) is to look at the change in performance for both BP and C++ in comparison to the performance you have when not running any AI simulations.

The second problem I have with your post is you are comparing the FPS numbers directly, whereas you should actually be looking at the computation time per frame.

I’ll first take a look at this:

When starting the simulation, the frame rates drops from 200+ to 160 fps for the C++ version and 132 fps for the BP version. We can consider this result as a 21% brute gain performance of using C++ over BP.

You claim that 160 / 132 =~ 1.21 means that there is a 21% performance gain for using C++ over BP (at least I guess that’s where the 21% comes from). But actually, we have the following:

• No Simulation: 200FPS –> (1000ms per second) / 200FPS = 5ms computation time per frame
• C++ Simulation: 160FPS –> (1000ms per second) / 160FPS = 6.25ms computation time per frame
• BP Simulation: 132FPS –> (1000ms per second) / 132FPS = 7.58ms computation time per frame

So, we already have 5ms of computation time per frame required for all the global systems implemented in the Unreal Engine. Your C++ simulation adds 1.25ms to that, and your BP simulation adds 2.5ms to that. This means that the C++ simulation is actually TWICE as fast (100%) as the BP simulation, not only 21% faster.

Now, you did mention that the FPS was capped at 200FPS, so maybe the FPS should actually be higher for the “No Simulation” case, so these results actually are kind of useless (I only looked at them because you did too in your conclusions). I’d say it’s better to look at the case of 2400 actors, because there we have a clean, non-capped FPS value even for the case without simulation.

• No Simulation: 95FPS –> (1000ms per second) / 95FPS = 10.526ms computation time per frame
• C++ Simulation: 71FPS –> (1000ms per second) / 71FPS = 14.085ms computation time per frame
• BP Simulation: 60FPS –> (1000ms per second) / 60FPS = 16.667ms computation time per frame

So, now we have that the C++ simulation adds ~3.56ms of computation time per frame to the baseline, and the BP simulation adds ~6.14ms per frame. That’s still 6.14 / 3.56 =~ 1.7 times as much computation time

Disclaimer: I only looked at this briefly before lunch, and am not an expert at benchmarking, so I may have made a mistake somewhere and would be happy to see it pointed out. But I still think this seems correct

[yamashi]

A correct approach to benchmarking blueprints vs C++ would be to make the engine’s impact on timings negligeable, by doing very intensive computation that last for multiple seconds per frame.
For example a code to do an n-body simulation like http://benchmarksgame.alioth.debian.org/u64q/program.php?test=nbody&lang=gpp&id=1

The benchmark posted here only shows that C++ is indeed faster than blueprints but nothing more.

[sylvain_l]

@
thanks for sharing your test results.

@DennisSoemers
I agree on your way for only taking into account the same computation to compare the performance for a pure statistical analysis BP vs. C++ for a piece of code.

But in the generic use of UE4 - creation of game - I think what finally matter at the end, for the player, is FPS, and the full computation time ! So dropping the “whole bunch of C++ behind the scenes” can’t be done in that perspective.
I have no idea how the “whole bunch of C++ behind the scenes” is scaling when you go from that test case to a full and complex game case. But I guess it’s still be a good part of computation time.

[DennisSoemers]

@sylvain_l;583582

@DennisSoemers
I agree on your way for only taking into account the same computation to compare the performance for a pure statistical analysis BP vs. C++ for a piece of code.

But in the generic use of UE4 - creation of game - I think what finally matter at the end, for the player, is FPS, and the full computation time ! So dropping the “whole bunch of C++ behind the scenes” can’t be done in that perspective.
I have no idea how the “whole bunch of C++ behind the scenes” is scaling when you go from that test case to a full and complex game case. But I guess it’s still be a good part of computation time.

@sylvain_l

That’s a fair point when the goal is to decide whether to use BP or C++ to implement something in your game. This is why I believe typically people mention in these kinds of “C++ vs BP” topics that, theoretically, C++ is faster, but it probably won’t matter for most things that people are implementing because most features that people implement have a negligible impact on the performance anyway. Any feature that is complicated enough to have a really significant impact on performance is probably gonna be too complex to implement in BP anyway without getting lost in the spaghetti (unless you’re purposefully trying to harm your performance, for instance by creating an infinite loop).

The benchmark that @ described does in fact provide evidence for this point, that you really have to go out of your way with huge numbers of actors to actually add a noticeable amount of overhead to what the engine is already doing, regardless of whether it is implemented in C++ or BP. But the conclusions as written in that post, describing a numeric value for the difference in computation time between C++ and BP, are not correct.

[motorsep]

@gameDNA studio;583310

Maybe on desktop difference is not so significant but on mobile my game made with BPs had ~20 FPS and now made with C++ has ~60 FPS so it’s about 3 times faster.

What if you use BP > C++ nativization option ? Would it still be ~20 fps ?

[gameDNA]

@motorsep;583613

What if you use BP > C++ nativization option ? Would it still be ~20 fps ?

I tried to use this feature but for my project it doesn’t work… Many errors I’ve already reported it to Epic…

[Anatol3d]

Blueprint Optimization | Live Training | Unreal Engine