Swarm agents refuse to render

I setup the slaves of our Renderfarm to help with light baking.
3 Slaves, Red02 to Red04, are setup properly and are oviously assigned to the host machine (HICCUP)
I hit “Build Lighting Only”, but Red01 and 03 refuse to start. I manage to render with 2 additional slaves once, but some always just idle. Any ideas?

BTW building takes ages, even with preview mode.