I did try updating our ffmpeg from 5.0 to 7.1.1 and we still see the issue.
I am currently hacking my way around it by using a QtCore.QTimer.singleShot to wait a certain amount of time and then calling
unreal.EditorPythonScripting.set_keep_python_script_alive(False)
to make the editor quit by itself.