I’ve been working on Unreal Engine for more than 2 years, I want to tell you how I feel about it: it’s crazy!
And I think all the craziness originated from the class reflection system, we all know C++ is fast but one of its drawbacks are it don’t have reflection system, but apparently Epic Implemented the Unreal Header Tool to scan your source code and generate reflection code during complication, I wonder is there any other engine did the similar thing but personally I think this is both scary and incredible.
and all this make it possible to create “dynamic class” or blueprint generated class, and to facilitate these you have to make your own object memory management since these class have variant members. I know reflection is useful for editor stuff but I wonder how other engine handle this beside use other language ?
Well, a lot of this goes back all the way to the beginnings of Unreal. A lot of what exists to support Blueprint was also here in some form or other to support the old scripting language in the old engines, UnrealScript.
I can’t speak for how much of what we have now is directly related to UE1, but I suspect there’s a surprising amount of the internals that can be traced all the way back.
So, this is 20+ years of development that we have here. It’s definitely pretty crazy. 
I’ve worked on several engines, and they all have some set of macros and templates that bind game-level (or scripting-level) “properties” to C+±level “struct fields.”
Exactly how it works varies a little bit, but they all solve the same problem, and thus they all end up sharing the same general shape.
And, because C++ doesn’t have any static reflection support in the language, you have to either do it with macros (most people do this) or use some kind of code generation or schema generation tool (fewer people do this) or a combination of both (Unreal does this.)
You may be familiar with a variety of such tools – boost::python, SWIG, tolua++, and so on. They all expose higher-level objects with reflected properties and some kind of introspection to make editors and savegames and networking work.
What I find crazy, or perhaps lamentable, is that the exposed object graph is so … class-based. There are deep inheritance hierarchies, where I’d really rather have composition-of-interfaces.
I appreciate your sharing. I think some form of reflection makes sense, since it’s the foundation of editor stuff and serialization, but I’m quite curious of how other engines do this. If one engine choose to do this in macro, I wonder if it would be inevitably more cumbersome to use than what we see in Unreal. and I’m curious about what other engines choose to use similar code generation technique? And if they do, are they all done in custom build process?
For C++: When you expose things with only macros and templates, without code generation, you end up having to repeat the name of each thing at least twice, because of how C++ compilation works. And you’ll have to remember to keep them in sync! (If C++ had static reflection, or even dynamic reflection through type_info, this wouldn’t be a problem, but no such luck …)
For most other languages: C#, Python, even Java support reflection, and most engines on top of those languages use that mechanism instead. Typically, you add decorators on top of the getters/setters/properties to tell the editor what to do and how to deal with it. Reflection often has performance implications, though. Compare how Unity games often use function names as messages and use reflection to discover them, which ends up costing a lot of performance.
If you need code generation, you need a custom build step of some sort, because C++ doesn’t have any support for code generation on its own. (As opposed to something like go:generate in go, for example.)
Although the idea of a build being “custom” only really comes from IDE based environments like Visual Studio and XCode, most other environments use some kind of makefile approach (cmake / ninja / bazel / make / jam / whatever) and thus everything is “customer” – which means nothing is “custom,” that’s just how it’s done!
I see, In terms of engine source code Unreal is my only experience, so I 'm always curious about how other engine do things, thanks again for giving me a clearer picture. You mentioned your preference on “composition” over “inheritance”, I have a different experience. when I work on ActorComponents more than once I found there are some tricky inter-component relationship that seems to violate the very goal of these component, the CharacterMovementComponent write on CapsuleComponent, then many other component will have to access MovementComponent, these dependency makes me deeply uncomfortable. Components are meant to be reusable and isolated, but these often unavoidable entanglement seems violate this principle. I wonder what’s your opinion on this matter? how would you tackle these dependency?
Engineering is the art of compromise. In this case, it’s the compromise between good performance on a variety of hardware, versus de-coupling between components.
It is possible to use an event bus type system to “broadcast” whatever data need to be updated about other components in an actor. Components would then register to “listen” to the broadcasts they are interested in. This is 100% de-coupled. However, there is also significant runtime overhead in sending all data through such a system, especially movement data that changes every frame. For an engine that targets some variety of hardware (used to be PS/3 and Xbox 360 with serial in-line cores, now it’s things like Switch on the low end) that cost can add up to lower frame rate.
Thus, the engineering trade-off here is that the network movement code and character movement component is more efficient, and the draw-back is that you have physical coupling to the capsule component.
Btw: always choosing the most general choice, or even always choosing the cleanest choice, is not “engineering” in this sense, because there’s no trade-off and analysis. Similarly, always choosing the dirty, or always choosing the runtime-fast, choice is also not “engineering,” again, because there’s no trade-off. The art lies in having an opinion on when to do which.