I had that same issue and in my case the problem was that the Sky Light had Cast Ray Traced Sadows Enabled. What solved the problem was to increase the Samples per Pixel in the Ray Tracing section of the Sky Light. I know you had this problem a while ago but I hope this helps someone with this same issue.