FPS drops to half when looking at furnished building in UE5

Okay, so I have two different archviz scenes in UE 5.0.3 and they are both consisting of interior as well as exterior. One is a lakeside villa with almost no artificial lighting but only sun and skylight. It is surrounded by lots of foliage and trees. The problem is that if I am looking in a direction where there are less meshes visible on screen, it’s fine but as soon as I look in the direction where many meshes are visible on the screen, it drops the frames suddenly and drastically ruining the interactive experience. I have lumen, raytracing, and nanite enabled and active in the scene. If I use DLSS, it helps me gain 10-20 fps in such areas.

The other scene has lot less foliage and lot less meshes but is experiencing same issue.
I am using 3070 TI on a B350 motherboard and ryzen 7 1700.

Earlier I used to bake in UE4 and didn’t encounter this issue.
I moved to UE5 for lumen and nanite

When you say ‘many meshes’ you mean the trees?

You could try forcing them up a LOD.