I had similar issue with my Skylight and realised that I forgot to add a Lightmass Importance Volume and it fixed it.