I tried increasing the gamma in the post processing settings. It did get brighter, but the HDRI contribution to the GI is still missing I think.
I usually use 8 frames of engine warmup for the MRQ and I can see lumen kicking in around the 4th-5th warmup frame, lighting up the scene. In case of the panoramic renderer, I dont see this happening at all.
In these images, I can see that even outside it is not lit up properly in panoramic rendering (but it is when rendering non-panoramic images):