Basically I think lightmass and Swarm rendering is pretty bad. I wrote a post about why, here.
Regarding Swarm Rendering in your case, it depends on the complexity of the scene. What you have done now is disable local cpus in exchange with network cpus. So there is zero speed gain. Because just like before the same count of cpus are working on the scene. Once your scene becomes more complex and your lightmap count is bigger than your local cpu count swarm rendering will kick in and speed up things. But also just to a certain level. On our renderfarm with 20 quad cores I measured a speed increase of 2-4 times, not 20 times faster as one would expect. But it depends on the scene. Usually for me it looks like this. Most cpu finish pretty quickly and just a few render to the end. One last thread takes forever 
As I´ve written in the post I linked. I don´t think Swarm Rendering is very efficient.
ps. don´t forget to set your local CPU count back 