I think the specular reflections are actually a bit cheaper than diffuse, because the cone angles are narrower. How big is a smaller scene, vs. a big scene, in your opinion? In other words, what is your target scene size and complexity? Outdoors with huge specular reflections of emissive mountains on water?