I was able to reproduce this and it seems that it is caused by using the Compile button in the editor rather than building in Visual Studio. A bug report was entered for this but the developers of this section of the editor have mentioned that this is the intended functionality. Please compile directly in Visual Studio instead.
Report: UE-50753