Does UE4 is the best engine? That’s kind of a loaded question to try to answer. Historically, UE definitely has been popular, at least in part because it is extremely flexible and pretty easy to work with. But UE3 has extreley high system requirements (rather odd, considering the first two were extremely scalable), so it likely won’t catch on for a while.
The vast majority of game though still use engines that are developed in house, rather than paying for licencing an engine such as UE. In some cases, these engines are written for a single game (and thus extremely streamlined for the needs of that game) and then discarded.
UE’s current competators would be Half-Life’s Source Engine and Cryware (the Far Cry/Crysis engine line). AD-X is a smaller one, but strangely popular with Capcom. SquareEnix is developing an engine formerly called the White Engine (don’t remember what they renamed it) for FF XIII, and plan on using it for every PS3 project they ever do, but no word on if they plan on licencing it.