The problem seems to be a bit different than what I thought. I put a sphere reflection capture actor and my reflecting sphere into a very small room and it still shows the sky.
So it’s not the view frustum of the reflection capture but the fact that somehow, for some reason, the sky sphere is superimposed on the reflection capture image.
If I delete the Sky Light from my scene, the reflection captures show the correct image, but my whole map goes dark (I’m using dynamic lighting only). How can I avoid this?