Just wanted to add this for anyone getting here via Google;
If you want to adjust the distance at which a light is visible, you can do so in the light properties under the Performance section. Specifying a Max Draw Distance will cull the light when you’re X distance away. This is really helpful for managing performance with dynamic lights and is disabled by default (set to 0). Setting it to anything above 0 will enable it, and you can adjust the setting in the viewport which is a really great way to make sure your lights don’t blink out when you can still see them.
Sadly, there doesn’t seem to be a way to set a max draw distance for all lights automatically so you have to set it manually on every light. You can select all lights at once and adjust though, so it’s something that can be done as a last step once your level is complete.