Found a solution for the issue I was having:
Just like lumen has issues with small emissive objects, ensure your cubemaps also do not have small bright pixels. I had left my sun enabled in my sky material, which created a very intense bright spot in my captured cubemap.
Capturing the cubemap without the sun resolved my issue.