Do you have raytracing shadows enabled inside the skylight? They’re disabled by default. (You may need to toggle realtime capture on/off to get shadows to show up… for some reason it seems bugged and shadows don’t appear immediately in some cases)
You’ll also want to make sure you set your raytraced max roughness to 1, otherwise you will get a lot of specular reflection bleed as the engine tries to blend in the reflection environment and to my knowledge there’s no way to turn this off. This can be mitigated to some extent by placing reflection captures but it is something you’ll need to be aware of.
Also your GI issue seems like it is an issue with either the exposure or brightness of your HDRI. An easy way to isolate the problem would be to use one of the built-in HDRIs that ship with UE4 (just search “hdr” or “cubemap” in the search section)
Also if your scene has reflection captures in it already, try rebuilding them
In my case, I have no reflection at all (specular = 0)
I was using physical camera values when setting up the environment, so my intensity (in the skylight component) was set high. As a test I used auto exposure and set my intensity back to the default 1, but the gi was still an issue.
As for the hdr image I am using, it is a default one (the one you get when you drag an hdri backdrop in)