Go to the details panel of your selected light. Under the transforms you’ll have the Mobility of the light: Static, Stationary, or Movable. Static and Stationary lights use Lightmass (prebuilt lighting) to get great looking shadows in the scene. So after you place a new light (which defaults to Stationary) you need to Build the lighting (look toward the top of the screen for a big button) in order to have it precalculate light and not act as an expensive dynamic light.