A lot.
Memory wise it likely destroys performance at the occlusion level.
Lighting wise the shading errors would be much more visible.
Instead of doing that, since there is really no limit to the size of anything if you do things right, find what size of 4 levels (landscape or mesh) you can keep comfortably loaded in your target machine for your heaviest game scenes.
Once you have that, just cut up the project in tiles of that size and set up a system to load/unload them as needed - be it world composition or custom.