For me it’s fairly simple; I get irritated as hell at things about UE4 (especially in the gameplay framework) almost daily,but, if it comes down to it, I can do my own implementation of features from scratch without working around a black box. It’s completely open.
Unity is a more controlled, funneled, and user-friendly toolset, really. It’s working on the extensibility aspect, but it won’t get there any time soon.
But, really, it’s just a matter of treating them as toolsets. Both have strengths and weaknesses. I know I could never have used UE4 as effectively as I am (for a one-person team)… Any time before I started using it, really. But it works for me and the way I like to work. Unity helped me get to that point.
The one thing I’d say is a major difference: I feel, sometimes, like UE4 has an editor based around its incredible engine. Unity’s editor is basically celebrating its twentieth wedding anniversary with the engine. They work together incredibly well. Which, I think, is where its user-friendliness comes from.
Whatever toolset works best for a team and the project is really all that counts.