I'm getting a "LowLevelFatalError [File:Unknown] [Line:153]" upon trying to launch my game after renaming the project. What am I missing here?

Yes, you should name your project with the same Unreal Engine naming rules when building the project. For example: there should be no spaces in the name, such as newgame not new game, no special characters, etc.