I might have found the problem. When using sequencer and a Cine Camera, it has a lot of the same settings the post process volume has, but it seems the camera settings work when rendering a scene, while the post process volume works on the real time display.
I noticed this when starting a new scene and switching the post process Exposure to manual and tweaking the camera settings. Only the rendered sequences got the changes, but tweaking the same settings in the post process volume changed the display in the editor.
It looks like my main problem was not having the viewport type set to cinematic viewport which seems to show what is rendered instead of the real time view.