Looking for improvements for Scene Capture 2D lighting/shadows

I’m using a Scene Capture 2D component to make a portal and I can’t get the shadows/lighting to look close enough to the player’s view that it’s not a jarring change in shadows, especially with metal/reflective materials. Anyone got any tips on getting it closer to the player view?

Scene Capture/Portal:

Player view after crossing:

I made sure the capture is using lumen and the capture source is the final color: