Download

Deforming skeletal mesh with object space normal map?

I’m new to the world of normal mapping. I’m looking at normal maps to fix some really bad mesh normals on a corrective blend shape.

Tangent space normals haven’t worked out, since they don’t override the underlying mesh normals. I’ve done some testing, and object space, by default, does not sound like it should be used on deforming meshes. Except for this on Polycount:

http://wiki.polycount.com/wiki/Normal_Map_Technical_Details

Rainbow colors. Objects can rotate, but usually shouldn’t be deformed, unless the shader has been modified to support deformation. Object-space is also called local-space or model-space.

I’ve searched around the UE forums/web and haven’t found anything on this. Is a material “fix” even possible to make an object space normal map deformation friendly?

If not, is there a way to completely override the mesh’s normals using a tangent normal map somehow?

check this
http://www.3dkingdoms.com/tutorial.htm
starts from “Light Vector Computing Per Vertex for Skeletons”.
What’s critical to use object space normal on a deforming mesh is to pass rotation of each vertex and add to the map.
im working on this too, but im really a noob. if you achieved it in unreal plz help me.

An object space normal map can’t be readily made to work with a deforming mesh. To make a simple example, think of a book laying open on the table. The pages are facing up and would have +Z normals, the covers are facing down and have -Z normals. Of you were to animate the book closing (such as through a morph target/blend shape), once the book has closed, and object space normal map would still be -Z on both covers, despite the fact that the front cover is now facing +Z. This is because OSN show the absolute normal vector of a pixel, instead of the relative difference.

My advice to a beginner would be to fix the geometry that’s causing you issues instead of trying to repurpose unsuitable tech.
Are there hacky ways you maybe could get it kind of working? Maybe. But it’s pushing a square into a round hole. And it could easily give you twice as many headaches as it fixes.

As to your last question. A vertex normal is just a vector. A tangent space normal is also just a vector. Using vector math, you can transform one vector into any other vector. You do not “override” the normal, you “augment” it, and there is no limit to how much you could theoretically change it. You just have to solve for how much you need to change the normal relative to the existing normal instead of just overriding it completely.

One brute force method would be to use an additional normal map alter the normals. A lerp could control when and if correction is needed, and that can be driven by a material parameter that is tied to the blend shape. When the blend shape is activated, the corrected normal map is blended against the original one. I believe this is how Rockstar games achieves the animated normals seen in their games. (See character shirts wrinkling as you run in GTA, or horse muscles flexing in RDR2).