Swarm Agent and Swarm Coordinator - CPU count tweaking?

Somehow, I get better build times when I am using 1 CPU ore less than the maximum core count available on the systems. Did anyone else see the same behaviour?