Light build takes hours with a Intel Xeon E5-1620 cpu

Hi everyone,
I discovered that on our machines with an Intel Xeon E5-1620 CPU light build takes hours, while on other machines with an Intel Core (i5 to upper) it just takes less than 5 minutes.
Inspecting the Swarm Agent during building I discovered that what takes hours is the Skylight Radiosity wich takes max 3 seconds on a Core i5, and hours on a Xeon E5.

Is there something I can do to speed up the process on the Xeon E5?

Considering the project and lightmass settings are the same, I tried to play a bit with process priority for Swarm Agent and Unreal Lightmass but with no results.
I plan to switch to shared build over a network using swarm agents on multiple machines, but in the meantime if there is something I can do to speed up light build on Xeon E5 I would really appreciate to know it.