I finally managed to fix the issue, I had to replace every occurrence of
Settings->SaveConfig(CPF_Config, *DefaultConfigFilename);
with:
Settings->TryUpdateDefaultConfigFile(*DefaultConfigFilename)
Not sure if this is a bug or a wanted behaviour but it took me way to much time to fix it!
This function actually seems better because it just updates the section that has been changed but I’m still confused on what changed during the update…