Am I the only one still having normal mapping issues in 4.4?

[Adam_Nerva]

I’ve been doing test bakes with xnormal and 3D-Coat, both using the mikktspace tangents, and while the results are mostly OK, I am still getting hard shading on flat quads where the tri-split is. The meshes are already triangulated before baking and import in UE4. I have read everything that Epic has posted on the subject, and tried every suggestion, but I still cannot get the kind of flawless normal mapping that I was able to achieve in Unity with handplane3D. I’m close to giving up on this, because as far as I can tell, it seems to be a bug in Unreal’s tangents calculations. They keep saying that it is fixed, but I’ve seen other users recently on the answer hub with the exact same problem I’m having. I am really hoping that Epic can get this situation resolved soon.

Answer hub thread where other users have the same issue:

https://answers.unrealengine.com/questions/14375/normal-map-workflow-with-xnormal-is-not-working.html

Because these artifacts are so noticeable, I am wondering what magical internal pipeline are Epic’s artists using that they are not suffering from this?

[SonKim]

Hi Adam,

Sadly, I’m not up to date with my UE4 sub(still on 4.2.1) so I won’t be much help. Are your UV split at hard edges? Also can you post a picture of what it looks in UE4 and a UV map screenshot. Can you upload the problem asset(low,high, cage)? for people to test?

[FrankieV]

Looking at the examples it looks like the smoothing groups are off. I’ve seen the same problem on geometry where the smoothing group was bonked and the triangulation would show up in the light map.

Can you post the normal map?

If it is an issue with the normal map then the flaw will show up clearly and if not then more than likely it’s a smoothing group problem.

When you export the model was included smoothing groups checked as to what needed to be exported?

This one caught me more than a few times and an edge cut like that does not really say an normal map problem to me.

[Adam_Nerva]

OK, here is one of my torture tests:

https://dl.dropboxusercontent.com/u/27648883/AdamsNormalTortureTest.zip

And here is the expected results in Toolbag2 vs UE4.4. Toolbag2 is properly synced to xnormal/mikk, and from my testing UE4 is definitely not. They just seem to be calculating their tangents differently.

Thanks!

[zeOrb]

@Adam Nerva;116183

OK, here is one of my torture tests:

https://dl.dropboxusercontent.com/u/27648883/AdamsNormalTortureTest.zip

And here is the expected results in Toolbag2 vs UE4.4. Toolbag2 is properly synced to xnormal/mikk, and from my testing UE4 is definitely not. They just seem to be calculating their tangents differently.

Thanks!

Normal map is horrible, look at triangles, they are EVERYWHERE ~.~. Of course you will get some weird result, you definitely should rebake it with proper settings.
However, your have problem with importing your normal map. UE4 has different compression settings for different types of textures and usually UE4 assign proper compression setting correctly. In your case UE4 assign TC_Default compression settings instead of TC_Normal because you have alpha channel in your normal map and you should not have it there. You can assign it manually in textures properties or you can delete alpha channel and reimport texture.

[Adam_Nerva]

Hey, thanks, good catch on the texture compression, that usually just works, I need to check that next time I am seeing shading errors. It looks much better, but still not correct. There is faceting that should not be there:

With regards to something being wrong with the normal map, I don’t understand what you mean. Triangles everywhere, huh? This is a low poly torture test. That is how the normal map should look.

Thanks.

[zeOrb]

@Adam Nerva;116217

Hey, thanks, good catch on the texture compression, that usually just works, I need to check that next time I am seeing shading errors. It looks much better, but still not correct. There is faceting that should not be there:

With regards to something being wrong with the normal map, I don’t understand what you mean. Triangles everywhere, huh? This is a low poly torture test. That is how the normal map should look.

Thanks.

Each lowpoly triangle is “highlighted” in your normal map, that’s why you see weirdness, just look at previous screenshot and compare to this rebake with different parameters

Your unwrap is designed with the idea that you have multiple smoothing groups and they are assigned to different sides of cube. One side = one UV shell and one smoothing group. If you will use single smoothing group with unwrap like this… you will get weirdness. You should either re-unwrap model to have less seams and rebake with single smoothing group or re-assign multiple smoothing groups, according to your UV shells.

[Adam_Nerva]

OK, yeah, that is what a normal map looks like with one smooth group, and most engines support this, even UE4 per the documentation:

https://docs.unrealengine.com/latest/INT/Engine/Content/Types/Textures/NormalMaps/Creation/index.html

You can use this method one of two ways. One is to give your model a single smoothing group. This will create a very gradient heavy normal map but few seams on UV edges. The other method is to break Smoothing groups along UV seams. This will create a flatter normal map but has some seams on UV edges.

The first step is to make sure your model consists of 1 smoothing group. It also helps to triangulate your model using a modifier, that way you do not have to worry about any application changing your triangles.

I view multiple smooth groups and supports loops as tricks that are needed when you don’t have a synced tangents workflow. These things should not be needed.

[zeOrb]

As far as I’m aware this style of unwrapping box was introduced due to the way how uv shells corresponds with smoothing groups. At least it is what I remember from thread at polycount, created by Marmoset Toolbag developer. There is like zero sense to unwrap your model like this and then use single smoothing group mesh for baking. It’s not a trick, it’s the way how calculation work, but you can view it however you want of course

Your normal map does’not work correctly in Marmoset Toolbag either(And yes, I flipped Y-axis and setup everything correctly). Increase reflectivity/specularity and you will see same triangles at different angles. Of course it still “works” in some cases, but it’s definitely not that quality you should aim for

https://youtube.com/watch?v=vVP0wEy37kw

[Adam_Nerva]

Hey man, I appreciate the help, really, but in a synced tangent workflow, one smooth group is all that is needed. The point of this torture test is to see if UE4 is truly synced to xnormal/mikkspace.

Regarding your Toolbag render there, you need to set the tangent space on the mesh to mikkspace.

[FrankieV]

Well a common problem when extracting a normal map with in 3ds Max is to many smoothing groups.

To bake a normal map in 3ds Max I usually set all the smoothing groups to 1, use a flat material with no spec or gloss and a single light set to ambient only.

This way only the diffused is baked with out lighting or artifacts being introduced by the material.

[Adam_Nerva]

Here’s another tangents torture test.

https://dl.dropboxusercontent.com/u/27648883/AdamsTortureTest-2.zip

It looks like UE4.4 is close, but something is still off. There are small anomalies. And I really have to watch that texture compression, it keeps defaulting.

[Adam_Nerva]

So it looks like the vertex normals in the FBX exports from 3D-coat aren’t being read correctly by UE4. There is a workaround by re-exporting from Maya. I’ve notified the dev. Other than that, 3D-Coat MikkTSpace bakes appear to be perfectly synced with Unreal 4.

Correct settings for 3D-Coat are:

Maya Normals calculation
3DS-Max, LW style normal maps
MikkTSpace tangents

[zeOrb]

Oh yes, maybe you are right about different tangent space!
However, I never experienced such problems, I bake in xNormal and SubstanceDesigner, but seems like due to different workflow everything is looks okay shrugs
About your last picture - I see same triangle artifacs as before

[Adam_Nerva]

@zeOrb;118065

Oh yes, maybe you are right about different tangent space!
However, I never experienced such problems, I bake in xNormal and SubstanceDesigner, but seems like due to different workflow everything is looks okay shrugs
About your last picture - I see same triangle artifacs as before

Yeah, the results from 3D-coat are not quite as nice as the xnormal bake:

But I just heard back from Andrew, the 3D-Coat dev:

But next beta build 4.1.12 offers solution for this - it may calculate normals and tangent space using any existing 3d modeler method. At least I think so This will need some testing. For now Maya (all vrsions) , Blender, Unity, Unreal Engine are supported, all other packages support will be done by users and I will add corresponding presets.

Also FBX export will contain complete set - normals + tangents.

So 3D-coat is going to have a UE4 baking preset. I look forward to testing it.

[Sladus]

I have issues with the weird shaders after rebuilding lighting on imported assets from 3d-coat as well. I hope the UE4 baking preset fixes the issues. That would be awesome.

[starseeker]

I recenty have issue with Normal also, but only in the recent update. Previously no issue.

There is discountinity at the seams in the level, anyone know what might be the cause. Its not lightmap for sure, as the seam are space pretty far from one another.