Right so why do you tie your visibility distance to the number of components rendered?
World composition and its level LOD generation and level streaming should help you.
While your whole world size is ~~32k x 32k, you can split it into several levels. For example, a grid of 16x16 levels, each with 2041 terrain(64 components). You are streaming in levels only around the player, so you would have to set up level LOD in a such way, that you only render 4 levels that are closest to the player at once.
So the actual terrain you are rendering is only 4082x4082(256 components). Levels, that are outside of these 4 levels, are replaced with static meshes LOD. So it takes only 508 (256+252) drawcalls per pass to render your world of ~~32k x 32k at resolution of ~~ 0.25 meters per vertex.
Actually, you could safely use a smaller level size at the cost of having more levels to deal with.
Starting point for world composition can be found here.
To sum it up, you should rely on distant terrain being replaced by optimized static meshes for huge world rendering.
Lastly, I would openly question your need to have such high resolution coupled with requested world size. Maybe you should rethink if you really need it.