In areas with very low indirect light you will get noise when not using FBRG, when using FBRG the result will be much smoother, but you may get blocky artifacts in those areas. You could turn it off and use the denoiser, but I wouldn’t recommend this unless you’re also turning lightmap compression off (probably a bad idea unless it’s an archviz scene) as the denoiser + lightmap compression produces patches of color banding.
Basically you have to choose between noise or blocky artifacts, in some scenes one might be better than the other. Turn FBRG on/off accordingly.