I’ll lay money it’s Lumen.
Do a GPU profile ( CTRL-SHIFT-, ) and take a look at the ‘big’ bits.
My framerate on one level went from 80 to 30. I fixed it using the Lumen overview. In my case, the surface cache was getting ‘too friendly’ with some hedges I had. I configured the leaves to not contribute to distance field lighting, and things were much more near normal after that.