Problem with switching to Oren-Nayar globally is that everything starts to look like plaster/sand (which is what ON basically tries to simulate - a very rough surface). MGS5 Has this exact problem where things can look really flat without a strong reflection/sweat etc.
On the other hand, lambertian can look a bit too “shaded”, but usually works better for slightly stylised/illustrated look.
So yeah, best would be per-material brdf but that would just kill performance.