It should not be affected by default. If skylight bounces can’t access some areas(caves for example) due to shadowing then these areas should look black.
What lighting system do you use?
Static with Lightmass or dynamic with Distance fields, movable lights and etc.