I managed to achieve the effect I was going for by using this going into the normal input and disabling tangent space normals. I also reset the normals in Blender and recomputed tangents to get rid of the morph target issues.
Here is the model with no morph targets.
With morph targets.




