How can I create upgradable buildings?

No one will be able to give you a short answer on that since you ask how to build a game.

Try breaking up the components you need, like how to model a mesh.
Then you need to learn how to spawn a instanced static mesh and replace with different meshes.
You also need to learn how to make widgets, how to spawn them where and when needed.

Google is a fine and quick way to search for tutorials. Good luck!