Possible Bug? Shadows Disappearing When using Fresnel Function in Masked Material

I’m not sure if this is a new bug but if not i’m surprised if this hasn’t been caught earlier - basically, when using a Fresnel function in a default lit masked material, I’m finding the shadows of my model are disappearing at certain angles in a very unusual way:

You’ll notice that this is not reflected in the actual model material in the world, here it behaves as expected.

Can this be fixed? right now it’s turning into a big problem for me :frowning:

Thanks.

Answerhub Post:

Followed answerhub post, might as well follow here just in case.
Would like to read potential reasons/answers.

Seems expected. As you are viewing the model with the light behind you, fresnel tones down opacity on the faces, that are more parallel to your line of view, and thus parallel to the light direction.
[SPOILER]

http://image.prntscr.com/image/a2744a88bcca450098318e47e5356685.png

[/SPOILER]
As you turn around, so that the light is roughly 90 degrees to your viewing line, fresnel again masks off the faces, that are closer to being parallel to the view vector, but this time unmasked portion of the faces, that the light sees is much thinner.
[SPOILER]

http://image.prntscr.com/image/682c73021d4b44b795d49bd255209f64.png

[/SPOILER]
When you are behind the object, while facing the light, the light sees no faces, as they are being cut by fresnel, thus almost no shadow is visible.
[SPOILER]

http://image.prntscr.com/image/154e82ec7980439f9b2c8ee325a17864.png

[/SPOILER]
So nope, that is not a bug.

EDIT
I think it might be possible to use custom node to force material portion to be executed only in shadow pass.

http://image.prntscr.com/image/bbe82959bbc34a08b6c50e1c25720cdb.png

Custom node contains:


#if(SHADOW_DEPTH_SHADER)
return fShdw;
#endif
return fAll;

Good suggestion.

You can also compute fresnel differently. This happens because fresnel is the Dot product of cameravector and normal. But camera vector does not get changed during the lighting pass.

If you compute the fresnel manually by computing camera vector first as normalize(CameraPosition - WorldPosition), that should also fix it because camera position DOES get updated to reflect the light position during the shadow pass. Give it a shot. Then you just need to add a Power and you will have similar control as the fresnel node.

edit since you are using the custom function for Fresnel, you can just plug in the custom camera vector right into the function also.

This seems to work! Is there a reason this isn’t a proper node in the shader editor?

Thanks Ryan, however I tired your suggestion and it doesn’t seem to work for me, shadows are still disappearing, perhaps I misinterpreted what you suggested?

Hmmm actually I may have jumped the gun. It looks like my suggestion works only in the vertex shader, but not in the pixel shader.

FWIW, that is how we got billboards in the kite demo to always shadow correctly. Otherwise, the shadow would be generated with the card facing you in the shadow pass, so if the light was 90 degrees, the shadow would be of a razor thin sheet. I thought it would also work in the pixel shader but I guess not.

You could probably make it work using the CustomUVs, but if Dethrey’s solution works, go with that. We could make that an official material function.

Just wanted to add that there is another potential solution I found based on your feedback Ryan:

a298e65acc0f47c7f4f772d02d7c9b4d2465b70c.jpeg

Here I plugged in a Vector3 (0,0,-1) and transformed it from view>world which seems to fix things also. Does this seem legit or a bit hacky?

Either way i’d definitely love to see the Shadow Custom code turned into a proper material function :slight_smile:

Edit: it seems this is a bit rough as the Fresnel will change depending on where the mesh is on screen. Any suggestions on how to improve it?

That last part is expected with this method since this will use the CameraForward direction. This kind of transform does work in the pixel shader so it will fix the shadow part, but it will also give undesirable results in the pixel shader.

The reason is because the ‘forward direction’ is only towards the center of the screen, so once things are off to the side, they won’t match that vector anymore. But CameraVector is the vector from the world to the camera, so it varies per position. This is the same thing that makes sprites appear to rotate in VR when you turn your head. To get around that I use vertex shaders to project them perpendicular to cameravector rather than just camera forward vector.

I see, out of curiosity, how would one make it work using the CustomUVs?

First, select the main material node to access the settings, and set “Number of Custom UVs” to be at least 3. You need 2 for this and likely want to leave CUV0 alone since it will actually change UV0.

Take the vector you want to pass into custom UVs, then do two Component Mask nodes. The first one is RG, the second one is B. Hook the RG to CUV1 and hook the B to CUV2.

Then you place down two TextureCoordinate nodes, one component mask node, and one append node.

Set one to UV1 and the other to UV2. Hook the UV2 one up to the component mask and select only R. Then append the UV2.x to the UV1.xy.

@RyanB

Could You kindly cover this in detail?
If that is the case, I would expect this network

http://image.prntscr.com/image/e39f8891e7854b06b1e538b22b53c1dc.png

To give different result in shadow pass.

But they are same:

http://image.prntscr.com/image/59318a5d2fa24361841c7e834c6f140e.png

Passing it via custom UVs has no effect.

TransformVector(ViewToWorld)

http://image.prntscr.com/image/2cd41d2ae679443caab500b8c7f96cee.png

works as expected(for directional light at least)

http://image.prntscr.com/image/faa6db83b4b94f9c87c61f436a814fe3.png

Ugg it seems like I’ve hit another snag - although the custom code node works for realtime lighting, BAKED static lighting still breaks when using the fresnel node:

df9d710e21f2957a9ad737e160a11009c26693da.jpeg

How is lightmass reading alpha when baking? Is there a way to avoid this in lightmass too? (baked lighting is also important to my project)

Thanks.

I doubt that it has to do with fresnel node. Most likely just low lightmap resolution. But in any case, there is a material expression named “LightmassReplace”. You can use it to exclude certain parts of material network from lightmass.

It definitely is caused by the fresnel node, lightmap resolution is very high (2048) on that floor, if I unplug the fresnel node and re-bake, the shadow looks fine.

However saying that, the LightmassReplace node works perfectly! Thanks so much!