How to use reflection captures in a "open world" enviroment?

Box reflection captures really improve some areas of my level, but Im having trouble figuring out how to use them without affecting other parts of the scene. My level is simply a building and an outside area. If I use one huge sphere reflection capture, It screws up the look of many materials. I tried One sphere and some box reflections but still get weird bright areas (like on the roof where one sticks out.

Is there a way I can tell certain materials to ignore reflection captures? Some of my materials look good with screen space reflections but not reflection captures.

Any insight on this would be appreciated :slight_smile: