Last I checked if you disable a plugin in a code project the plugin will not be built with the project. But you have to build your project not UE4 because UE4 will still build all plugins for the platform.
Also from my experience deleting a plugin from Engine/Plugins usually doesn’t do any harm. And if you need that plugin later you can restore it from your version control or from the original version of the engine.