I totally thought the same thing!
I get what OP means though, I’ve struggled with this ever since I started using PBR workflows, especially on flat surfaces like dirt and grass. I’ll usually just lower the specular value in the material but I know that’s not correct.
Does anyone have more info on what @Deathrey Mentioned regarding a compensatory factor based on roughness? Is this something that could be done at the material level or would shader code need modified?