I’m also having some issues with this and it seems that ‘build light’ does this for SSS (Subsurface scattering) materials. I found a ‘workaround’ that should help though. Change the mesh Mobility to stationary so the light pass wont be baked and the wierd shadows wont appear:
Select the mesh → Details → Transform → Mobility
→ Stationary