What is the most lightweight way to write and compile code?

I’m not sure if this is right, but
from what I’m reading around the web
I’m getting the feeling that in order
to work with code in UE4, I may not in
fact have to compile the engine.

Is this right?

Right you not have to compile the engine to work with code. Until you are planing to modify the engine.

I’m working on a project that contains
some code, what is the easiest way for
me to alter and compile that code?

I suggest, use the binary version of the engine. Which you get from the Epic Game Launcher->Library. And to work with code under Windows use Visual Studio 2013. My latest news is that VS Solution creation for UE4 in special 4.8 do not work with VS2015. The support for VS2015 will come with the UE 4.10 engine version. Under OSX you will work with the UE4 Editor and XCode. Regarding Linux I don’t know. A common workflow under windows would be. Create a Code Project from the UE4 Editor. The UE4 Editor will then open the VS Solution, you will compile your first code. From this point you can modify you code in Visual Studio or other Text Editors (But I still sugest VS) and just press the compile button in the UE4 Editor to compile your code and hot reload it into the editor. So… Edit code in Texteditor/VS, Start compile from UE4. To debug your code, you have to lauch your UE4 Project from Visual Studio debugger instead the Epic Game Launcher.

Do I have to build the engine, or can
I get away with skipping that part?
I’ve seen people talk about ‘code
projects’ vs ‘building the engine’,
but not totally sure what that means.

As all answer right away say, you do not have to build the engine if you use the engine binary from the Epic Game Launcher. And to clarify that “code projects” and “building the engine” means
“Code projects” are projects there the developers write C++ code to extend your game/project.
You don’t have to write code in a game/project, because a lot in UE4 can be done in Blueprint the Epic visual sripting language. But as you say you use code you would use a code project.

“Building the engine” is if you grab the code base from GitHub(not the epic game launcher) and compile/build the complete engine code by your self. As written I didn’t suggest it until you want to change the engine. Extending the Engine with modules is a other topic…