Hi vle07,
This sounds very similar to a memory leak that we are working on (UE-27195). It occurs when your level is not enclosed or the empty background is not occluded in some way. Levels with Atmospheric fog or a SkySphere do not encounter this issue, but an empty level will.
Can you try dropping the default SkySphere into your level and let me know if it still occurs when your camera is inside of it?
-.