I´v´e imported some 3-4 million poly´s terrains, but what kills the framerate is trees 
Even if use height-maps the vegetation is a challenge, and ground plants like grass perhaps the most difficult in large scenarios.
This scene has 45 000 trees, and it went from 70+ fps to 30-50 even if not many in sight, I only have 1 LOD so that might be the reason though.
://s28.postimg.org/jjdac8uvt/test4.jpg