Normal map messed up when imported at runtime

[uced]

Hi,

When importing normal maps at runtime they look like this:

And when importing the files in the editor and assigning them manually in the Material Instance, they look correct:

Both images are made with the same files, same parameters, yet the first one looks all wrong.

The only difference as far as i know is that in the first one the texture was imported at runtime.

Here is what i d. It’s a bit tricky because the mesh are already assigned a Material Instance, so:

// i first create a MID from the parent material
DiceMaterialInstance = UMaterialInstanceDynamic::Create(DiceParentMaterial, this);

// then assign it to the mesh
DiceStaticMesh->SetMaterial(0, DiceMaterialInstance);

// then get all the parameters from the original Material Instance (i checked in many ways that they are copied correctly)
DiceMaterialInstance->CopyMaterialUniformParameters(DiceMat);

// then assign the new textures
DiceMaterialInstance->SetTextureParameterValue("DiceNumbers", TexturePair.LayoutTexture);
DiceMaterialInstance->SetTextureParameterValue("DiceNormalMap", TexturePair.NormalTexture);

I checked that all parameters are correctly set with things like this applied both on meshes having the original MI and meshes having my home made MID, the numbers are the same:

GEngine->AddOnScreenDebugMessage(-1, 10.f, FColor::Red, FString::Printf(TEXT("SetupTexture Roughness: %f"), DiceMaterialInstance->K2_GetScalarParameterValue(FName("Roughness"))));
GEngine->AddOnScreenDebugMessage(-1, 10.f, FColor::Red, FString::Printf(TEXT("SetupTexture Specular: %f"), DiceMaterialInstance->K2_GetScalarParameterValue(FName("Specular"))));
GEngine->AddOnScreenDebugMessage(-1, 10.f, FColor::Red, FString::Printf(TEXT("SetupTexture Metallic: %f"), DiceMaterialInstance->K2_GetScalarParameterValue(FName("Metallic"))));

I tried to multiply the normal maps by various numbers with no visible result.
I also tried to invert some channels with no improvement.

So on the paper everything looks fine, the files are the same, the numbers are the same, but it’s all messed up when done at runtime.

I don’t know what it says about normals when it looks like this. Too strong ? Too small ? Wrong direction ?.

I already saw similar things in the past but it was always that the normals were incorrect on the mesh, which is obviously nothe case here.

Anyone knows a direction i could try ?

Thanks
Cedric

[NilsonLima]

How the texture node is set inside the material graph?

[uced]

Here’s the parent material:

[NilsonLima]

Thats OK, there is no TextureCoordinate node set on DiceNormalMap and DiceNumbers, but has on DiceSurface_BaseTexture and DiceSurface_NormalTexture, and not sure if TextureCoordinate has something fed into the details. Also, Id like to see the details set for the normal map nodes (Param2D).

[uced]

There you go:

[uced]

Oups, forgot the text coord:

[NilsonLima]

Let me check something, but maybe you have found a bug. I will check if I can suggest a different setup for your material that will prevent it once I have confirmed it. Just let me know which engine version you are using and also if Material Layers (experimental) is enabled or not.

[uced]

I’m using 4.20 straight from the launcher (no code modification nor recompilation) and the feature is disabled:

[uced]

Hi,

I found something super strange.

I thought there could be something wrong in the way i was importing the png picture of the normal on the hard drive and transforming it into a UTexture2D (compared with importing the picture in the editor), so i made a widget able to display a UTexture2D and displayed both normal maps.

It appears there is a difference, but the result is the opposite of what i was expecting:

As you can see:

• the normal maps that gives a correct die is all yellow (this is the one coming from the editor)
• the normal maps that gives a incorrect die has the actual correct light blue color of a normal map (this is the one loaded at runtime)

So the correct map gives a wrong result and the correct result is given by a yellow map !

To me the mystery is complete, but at least i’m glad i found a difference i can work on.

My problem is now to understand if i must correct my ingame imported normal map to make it yellow, or if UE4 works with a wrong normal map but somehow compensates it.

I’ll keep this thread updated if i understand something.

Cedric

[uced]

Quick update with a few facts:

• Forgot to mention but i’m positive my widget displays colors correctly (any other picture comes with the right colors)
• In the editor, when opening the normal map texture, its colors are correct (blueish)
• I just checked the rgb for that light yellow and it’s about (1,1,0.5).
• I know from theory that the blueish color of normal maps is (0.5, 0.5, 1)

So from my current understanding (which may be all wrong), it seems that at runtime, UE4 somehow applies a (1.5 -x) function to the normal maps before using them.

It could be a (one-minus + 0.5): inversion and a shift of origin.

At least it’s what it looks like with these particular values, but it might be a more complicated shifting/rescaling in the more general case.

As the end result is perfectly correct, i assume it’s by design but it seems very strange.

Cedric

[NilsonLima]

How did you generate the normals? It is important because:

Theory says about normal map that you have each RBG on the pixel as a vector pointing the direction the pixel is facing, so as RG is origin (x,y) and B is the head (z) you will indeed find more of a blueish color on the normal map.

If the normal map is an array of vectors for each pixel, having this information compressed or in a format with data loss is not good. I think there is the following checklist for you:

• check if when generating the PNG file it is selected any compression (remove the compression if there is any)
• check if you can generate in TGA format (try if the above results in the same issue you are facing)
• when doing the texture generation in runtime certify you don’t have compression
• if you are using Substance Painter, certify the normal map type you are exporting is the DirectX Normal Map type.

Let me know your findings after those! Cheers!

[uced]

Hi !

Thanks for the suggestions !

No success for the moment, but quick update with a few info.

Unfortunately i can’t use TGA because the UE4 image wrapper doesn’t support it:

IImageWrapperModule

EImageFormat

So i’m now using bmp.

The first tests above were made with normals from an online site.
But as things got serious, i did the normals with Bitmap2Material (from Allegorithmic). The results are the same but i can now be sure that:

• i export bmp with no compression
• i use a directx format

So i’m pretty sure my source is correct.

The problem i think i how i import it ingame.

Essentially here’s what i do (not putting every details, just the skeleton):

// create a wrapper module
IImageWrapperModule& ImageWrapperModule = FModuleManager::LoadModuleChecked<IImageWrapperModule>(FName("ImageWrapper"));

// get the type (i checked that it was bmp)
EImageFormat ExtensionType = ImageWrapperModule.DetectImageFormat(TextureData.GetData(), TextureData.Num());

// create a wrapper for bmp
TSharedPtr<IImageWrapper> ImageWrapper = ImageWrapperModule.CreateImageWrapper(ExtensionType);

// load data in the wrapper (it's compressed but we will get uncompressed later)
ImageWrapper->SetCompressed(TextureData.GetData(), TextureData.Num()

// create a container for the uncompressed data and fill it with raw data (24 comes from GetBitDepth() so i'm sure of it - but i also tried with 8 bit images with no luck)
const TArray<uint8>* UncompressedBGRA = NULL;
ImageWrapper->GetRaw(ERGBFormat::RGBA, 24, UncompressedBGRA)

// create a fresh texture
MyTexture = UTexture2D::CreateTransient(ImageWrapper->GetWidth(), ImageWrapper->GetHeight(), PF_B8G8R8A8);

// fill the texture with raw data
FMemory::Memcpy(TextureDataFromFile, (*UncompressedBGRA).GetData(), (*UncompressedBGRA).Num());

I think the problem is in the creation of the texture, when i specify a texture format.
With PF_B8G8R8A8 i get the initial wrong result (purple normal map that looks wrong on the dice).
I tried a few combinations with RGBA and BGRA image formats.
I can get various results with other format, but there are A LOT of them and i didn’t test them all

EPixelFormat

I looked in the editor, the texture says it’s compressed in DTX5 or BC5, but i get a crash when i try those.

So there i am, in short:

• my source seems good
• i use it uncompressed
• there are lots of available texture formats, some of them crash the game
• the formats suggested by the editor crash the game

I’ll go on with my trials/errors and will keep this thread update if i make any progress

Cheers
Cedric

[uced]

Just found this function: UTexture::IsNormalMap()

It returns true for the yellow normal map (from the editor = good dice picture) and false for my home-imported normal map.

So definitely something going on there, it seems that at runtime the game expects data compressed in a “normal map” way.

I guess now i have the raw data, i have to compress it into a DXT5 or BC5 format.

To be continued…
Cedric

[NilsonLima]

Where did you find about that piece of code where you create the texture?

I mean, the good source of such type of code would be the portion of code directly inside the engine source code @ the editor import operations (by drag or filesystem detection) when the asset is an image. Just curious and I would look at there to certify how it is done. The point is that such digging takes too much time, so if your source is already some part of the engine code, you would just submit a bug report with a sample project showing your issue.

[uced]

It might be engine code inspired, but it’s not mine: it’s from a function i found somewhere on the answerhub, but it was a very long time ago, like 2 or 3 years.

I have been using this bit of code since forever in various projects for usual pictures with no problem and now have to make a bit of digging because of this normal map problem.

I tried to find where it’s done in the Engine code but i’m not familiar with it and couldn’t find anything interesting at the moment.

But i just found this:

Is texture compression at runtime supported?

And it seems that texture compression is not allowed during runtime.

So now i’m not even sure what i want to do is even possible.

My current guess is that the game is packaged with compressed texture that the runtime engine can use.

So it would seem that by design we’re not supposed to create/modify normal maps at runtime.

I’ll give it another few hours and see if i can decide if it’s worth more time or if it’s a dead end…

Cedric

[uced]

Hi,

I couldn’t find anything new significantly interesting, i’m afraid it’s going to take me a lot more time than expected and it’s not that critical for me, it’s just normal map on small dice faces, i can live without it

So i’m going to skip this one for the moment and focus on more important issues in my project.

Anyway, thanks a lot for your time NilsonLima, your clever suggestions allowed me to clarify this issue and understand many things. Very appreciated !

Hopefully this thread can be a good starting point for others in the future who might be interested in this sort of things. I’ll certainly come back to it one day.

Thanks again

Cedric

[NilsonLima]

You are most welcome! Any idea or fact I might cross I will post here aswel!

[anonymous_user_bf9de05a]

Just wanted to post saying I’m having the same issue, in 4.20.

My workaround as of now is this method:

https://forums.unrealengine.com/development-discussion/rendering/104120-how-to-convert-bump-map-to-normal-map

It works great for what I need, since I was taking zDepth renders from vray in Rhino and making those normal maps in Photoshop. Now I can just use the zdepth map as the texture Object Parameter, but it seems like the normal maps I already made work as “height maps” as well.