Explain to me how modding the engine works, is my idea possible?

C++ and HLSL experience are what is needed. Since you have the full engine source code, you can do anything with it really if you know how to read it.

For modifiable terrain, it is currently only supported in the editor. However, since you have the entire source code available, you can easily extrapolate the code yourself and find a way to add it to the main game if you so desire. Or you can come up with your own solution and add it in, either way works!

For the networking, UE4 already has a pretty robust network system. You could add to it if you like, or if you really prefer, rewrite it from the ground up. Everything in Unreal is done with modules/plugins, so you can easily add/remove code that you use between projects.