Are your materials complex?
You could try to force not baking materials inputs and see if that does what you want.
