UIExtensions without plugins?

Editor it self runs on UE4 they are not seperate, all editor APIs are accessible from any point, ofcorse when games get package those APIs are unavailable

Modules are equile and can do the same things regardless where they are engine source, game project or plugin they will work the same, only difference is distribution, so it’s ok to do editor module in game project if you make game specific thing, but if you want to make something that works in every project, plugin is the thing.

In game project insted of .uplugin you add modules to .uproject file, structure is the same as in uplugin

With editor code, best practice is to have editor code in sperate module and mark it as "Type": "Editor", but if you really can’t seperate the code you use WITH_EDITOR, or mix of two, generly editor only class should be in sperate module.

As for how to start first you need to learn about Slate, editor UI framework… which also powers UMG and its avable in client to make game ui:

Slate User Interface Programming Framework for Unreal Engine | Unreal Engine 5.1 Documentation
A new, community-hosted Unreal Engine Wiki - Announcements - Epic Developer Community Forums

Also watch this, it a nice start point:

check up plugin templates as they show simple examples. Other then that look up engine source code, plugins source code to see how things you see in editor look like in code.... sadly thats only way to learn editor devlopment