When renaming material used in Mutable setup and solving redirector afterwards, the CO asset is requested to checkout, marked as dirty, but the change is not handled correctly. The material definition is lost.
It is expected that the material in the setup is redefined correctly.
Following your repro steps, I was unable to recreate the issue in 5.5. Is there anything else that you can share (assets, test projects, images) that might help? Did perforce block updating the file?
In terms of where you could start debugging, you could put a breakpoint in FAssetFixUpRedirectors::ExecuteFixUp and see if your asset gets updated and saves.