Unreal 5.1: Materials don't load properly on level change, but they work if the map is played directly

If you’re using a custom material for everything that is not loading properly (I.E. materials that are not from starter content) then just find every texture that is not loading, and open them, after that, there should be a button on the top of the texture editor that says “Reimport” just click that. (i had this exact problem)

Hopefully this helps!