For anyone who has the same problem, I found the answer. Reading the documentation I thought that lumen would work very easily with any landscape but depending on the size and resolution it can cost alot.
Deleting or changing the resolution + landscape LOD settings for my terrain got me, with lumen, from 40fps to lock 60. For hardware RT I got from 8 fps to 60fps aswell.
this helped me out
