You can’t load things in and out of memory like that, since you don’t know what might affect the lighting of something else, if you only load part of the scene into memory and render the lighting for that you may miss something that affects those objects.