Hi Tiago,
Was not expecting my comment to summon someone from Epic itself, thank you so much for your swift reaction, and I apologize for unnecessary gray hairs if you got any, because I got.
It was a bit unfair to me to say that issue is coming from the engine, it is just 5.4 update made it more prominent.
You can see stat scene rendering in my original comment and it shows that the issue is in GatherRaytracingWorldInstances.
So it all boils down to checkbox “Visible in Raytracing” on foliage.
I total have 560k foliage instances on my scene, and as soon as Visible in Raytracing checkbox is activated on foliage I see how this function goes crazy, which is natural for 560k instances.
It’s just that in 5.3 performance of that function was under my target value ~22ms, so I was not paying attention, and in 5.4 it stopped being there, and this drop is more noticeable the more foliage I have on my scene.
My workaround is obviously to disable Raytracing on foliage, because it is unnecessary.
However, I see that even if I set Cull Distance for foliage to be extremely low, function is takes performance. I don’t know much about it, but maybe that information will help you as well.
Extra unrelated side note: I am using Dynamic Grass Plugin https://www.unrealengine.com/marketplace/en-US/product/dynamic-grass-system, and that plugin doesn’t respect Visible in Raytracing checkbox, and that is what made my issue actually noticeable.