I’ve stumbled upon this same question. My current bottleneck is foliage and I’m currently playing around with different foliage cell sizes and HLOD bakes, but still no success. I’m using an open source version of 5.3 and considered customizing the code for RemoveFromWorld too.
Did you go through with it?