Yes, you have to rebuild the entire engine, not just the Niagara plugin.
You also can’t copy the Niagara plugin to your project as it needs access to engine modules that it won’t get from your project folder. It would also break dependencies from other plugins and modules to Niagara.