For anyone else trying to figure out why, “Cooking on the fly when launching from the editor” needs disabled in Project Settings - Engine Cooker. The engine returns a ValidationErrors > 0 since cooking on the fly needs disabled in shipping mode. Found the cause when debugging inside the UE5 source code.