Download

[UE4.24] Implement(Fix) Thick Glass Refraction and Double Reflection in Ray Tracing

I want to simulate scene with a lot of glass and generate dataset, but I found there are many problems about UE4.24 raytracing. I understanding they are still developing and some bugs are features for better performance. In this topic, I will offer ways to solve some problems. I hope it could help somebody.

At first, I list what problems will be solved in this topic:

  1. How to set IOR(Index of Refraction) of glass(transparent object);

  2. Fix thick glass refraction error. (Thin glass only need to set IOR as 1.0)

  3. Implement thick glass double reflection.

  4. Implement multiple times reflection of glass. (It will cause serious performance problem, so be careful to use it)

  5. Set IOR:

Before setting correct parameters in material, there is no refraction:
IOR_Ori.jpg

Ray tracing in UE4 actually uses “Specular” as IOR. We can use formula: Specular = sqr((IOR-1)/(IOR+1))/0.08 to convert IOR into specular. As follow, I create a material function:
IOR2Specular.jpg


“RayTracingQualitySwitchReplace” is a switch function which will choose RayTraced input as output in ray tracing mode, otherwise will choose Normal input.

Modify material as follow:
MB_Glass.jpg


Specular input doesn’t need Fresnel to adjust IOR.

Then, the refraction works and likes a solid glass ball.
IOR_New.jpg

  1. Thick glass refraction:

Because solid glass ball is hard to prove correctness of refraction. So I choosed thick flat glass which we can see it anywhere.
As follow image show, the refraction is error because it scale the chair so much.
FlatGlass_Ori.jpg FlatGlass_Ori.jpg

Then, we need to modify the shader file in UE4 to fix the problem.

  1. Add defination at the top area of file “RayTracingPrimaryRays.usf”:

#define MAX_IOR_STACK                3

  1. In the initialization part of RAY_TRACING_ENTRY_RAYGEN function, we need to define variables:

 float IorStack[MAX_IOR_STACK];
 int StackIdx = 0;

  1. Depend on the front/back face information, save/load IOR in stack before the call of RefractRay function:

if (bIsEntering)
{
        // Push Ior2 into stack
​​​​​​​        if (StackIdx < MAX_IOR_STACK)
​​​​​​​        ​​​​​​​        IorStack[StackIdx] = Ior2;
​​​​​​​        ​​​​​​​        StackIdx++;
 }
else
{
​​​​​​​        // Pop stack
​​​​​​​        StackIdx--;
​​​​​​​        if (StackIdx < MAX_IOR_STACK && StackIdx >= 0)
​​​​​​​        ​​​​​​​        Ior1 = IorStack[StackIdx];
}

  1. In RefractRay function, when we calculate Eta variable, it should be modified as follow:

/* Origin
float Eta = Ior1 / Ior2;
if (bIsEntering)
{
       Eta = 1.0f / Ior1;
}
*/

float Eta = Ior2 / Ior1;

After modification:
FlatGlass_New1.jpg FlatGlass_New1.jpg
FlatGlass_New2.jpg FlatGlass_New2.jpg
We can see the chair only offsets a little and nearly no scales, which likes the real world glass.

  1. Thick glass double reflection

In real world, many glass has double reflection, but in UE4 glass only has one reflection.
RealWorld_DoubleReflection.jpg


DoubleReflection_Ori.jpg

There are serval methods to solve and I saw someone suggest use two side material. However, the two side material will affect refraction, because in shader code it would make we cannot get correct front/back face information.
My method is also modification of shader.

  1. In “RayTracingPrimaryRays.usf” file, after done once ray tracing (in other words after the call of TraceRayAndAccumulateResults function), we add:

// Correct surface normal when hit a back surface
if (dot(-Ray.Direction, Payload.WorldNormal) < 0.0f)
{
       Payload.WorldNormal = -Payload.WorldNormal;
}

After modification:
DoubleReflection_New.jpg DoubleReflection_New.jpg
The red ball and the bookcase were reflected twice in thick glass.

  1. Multiple times reflection of glass

Before modification:
MultiReflection_Ori1.jpg


MultiReflection_Ori2.jpg

This part need to code a lot in “RayTracingPrimaryRays.usf” file, so I only breifly explan how I implement it and attach modified “RayTracingPrimaryRays.usf”.

In the original codes of RAY_TRACING_ENTRY_RAYGEN function, it only does once reflection in every ray (the reflection does not recurse). And here only recurse the transparent face.
What I do is using stack to implement recurse of ray tracing both on reflection and refraction, because HLSL doesn’t support recursion of function.

In my attached usf file, I used macros to swith my modification:

#define FIX_IOR 1
#define FIX_DOUBLE_REFLECTION 1
#define MAX_IOR_STACK 10
#define USE_ALTERNATIVE_RENDER 1
#define MAX_ALTERNATIVE_NUM 3 //alternative between transparency object and mirror object, be careful about the 2^num rays will be detected
#define MAX_ALTERNATIVE_STACK 10

After modification:
MultiReflection_New.jpg

After all modifications, here I post comparison of the refraction of glass ball between UE4 and real world:
RealGlassBall.jpg

Attachment: RayTracingPrimaryRays.txt (If you want to run it, you should change the file extension as usf and replace the original one)

I don’t know why my attachment cannot be downloaded.
Here I post my modification of multiple times reflection of glasses. All codes are in “RayTracingPrimaryRays.usf”.

  1. Defination


#define USE_ALTERNATIVE_RENDER        1
#define MAX_ALTERNATIVE_NUM            3        //alternative between transparency object and mirror object, be careful about the 2^num rays will be detected
#define MAX_ALTERNATIVE_STACK        10


  1. Struct for store ray tracing variable (used in stack)


#if USE_ALTERNATIVE_RENDER
struct FAlternativeData
{
    RayDesc Ray;
    FRayCone RayCone;
    float3 PathThroughput;
    float LastRoughness;
    float LastIor;
    float IorStack[MAX_IOR_STACK];
    int IorStackIdx;
    bool bHasScattered;
    int AlterSelect;
    bool bReflection;
    int AlterTimes;
    int RayTimes;
};
struct FAlternativeParameters
{
    bool bAllowSkySampling;
    bool bSkyLightAffectReflection;
    float3 Scatter;
    float Depth;
    RandomSequence RandSequence;
};
#endif //USE_ALTERNATIVE_RENDER


  1. The implementation of AlternativeRayTrace function for alter reflection and refraction (use Stack to implement Recursion)


#if USE_ALTERNATIVE_RENDER
bool CheckNextDataValid(FAlternativeData Data, int StackIdx)
{
    if (StackIdx < MAX_ALTERNATIVE_STACK && Data.AlterTimes < MAX_ALTERNATIVE_NUM && Data.RayTimes < MaxRefractionRays &&
        (Data.PathThroughput.x > 1e-2 || Data.PathThroughput.y > 1e-2 || Data.PathThroughput.z > 1e-2))
    {
        return true;
    }
    return false;
}

// Return Final PathRadiance
float3 AlternativeRayTrace(FAlternativeData Data, FAlternativeParameters Params)
{
    FAlternativeData Stack[MAX_ALTERNATIVE_STACK];
    int StackIdx = 0;
    Stack[StackIdx++] = Data;

    float3 PathRadiance = 0.0f;

    while (StackIdx > 0)
    {
        Data = Stack[StackIdx - 1]; // Stack Top

        // 1. trace ray and get pay load
        const uint RefractionRayFlags = Data.bReflection ? RAY_FLAG_CULL_BACK_FACING_TRIANGLES : 0;
        const uint RefractionInstanceInclusionMask = RAY_TRACING_MASK_ALL;
        const bool bRefractionEnableSkyLightContribution = Data.bReflection ? Params.bSkyLightAffectReflection : true;
        float3 PathVertexRadiance = float3(0, 0, 0);

        FMaterialClosestHitPayload Payload = TraceRayAndAccumulateResults(
            Data.Ray,
            TLAS,
            RefractionRayFlags,
            RefractionInstanceInclusionMask,
            Params.RandSequence,
            MaxNormalBias,
            ReflectedShadowsType,
            ShouldDoDirectLighting,
            ShouldDoEmissiveAndIndirectLighting,
            Data.RayCone,
            bRefractionEnableSkyLightContribution,
            PathVertexRadiance);
        Data.LastRoughness = Payload.Roughness;

        //
        // 2. Handle no hit condition
        //
        if (Payload.IsMiss())
        {
            // Pop Stack
            if (Data.bHasScattered && Params.bAllowSkySampling)
            {
                // We only sample the sky if the ray has scattered (i.e. been refracted or reflected). Otherwise we are going ot use the regular scene color.
                PathRadiance += Data.PathThroughput * GetSkyRadiance(Data.Ray.Direction, Data.LastRoughness);
            }
            if (!Data.bHasScattered)
            {
                PathRadiance += Data.PathThroughput * Params.Scatter;
            }
            StackIdx--;
            continue;
        }
        float3 HitPoint = Data.Ray.Origin + Data.Ray.Direction * Payload.HitT;
        float NextMaxRayDistance = Data.Ray.TMax - Payload.HitT;

        //
        // 3. Handle surface lighting
        //

        float vertexRadianceWeight = Payload.Opacity;    // Opacity as coverage. This works for RAY_TRACING_BLEND_MODE_OPAQUE and RAY_TRACING_BLEND_MODE_TRANSLUCENT.
        if (!Data.bReflection)
        {
            // It is also needed for RAY_TRACING_BLEND_MODE_ADDITIVE and  RAY_TRACING_BLEND_MODE_ALPHA_COMPOSITE: radiance continbution is alway weighted by coverage.
            // #dxr_todo: I have not been able to setup a material using RAY_TRACING_BLEND_MODE_MODULATE.
            PathRadiance += Data.PathThroughput * PathVertexRadiance * vertexRadianceWeight;
        }
        else
        {
            PathRadiance += Data.PathThroughput * PathVertexRadiance;
        }


        // Correct surface normal when hit a back surface
        if (dot(-Data.Ray.Direction, Payload.WorldNormal) < 0.0f)
        {
            Payload.WorldNormal = -Payload.WorldNormal;
        }


        //
        // 4. Handle reflection tracing with a ray per vertex of the refraction path
        //

        // Shorten the rays on rougher surfaces between user-provided min and max ray lengths.
        // When a shortened ray misses the geometry, we fall back to local reflection capture sampling (similar to SSR).
        const float LocalMaxRayDistance = Params.bAllowSkySampling ? 1e27f : lerp(TranslucencyMaxRayDistance, TranslucencyMinRayDistance, Payload.Roughness);
        if (Data.AlterSelect < 1 && Payload.Roughness < TranslucencyMaxRoughness)
        {
            // Trace reflection ray
            uint DummyVariable;
            float2 RandSample = RandomSequence_GenerateSample2D(Params.RandSequence, DummyVariable);

            RayDesc ReflectionRay;
            ReflectionRay.TMin = 0.01;
            ReflectionRay.TMax = LocalMaxRayDistance;
            ReflectionRay.Origin = HitPoint;
            ReflectionRay.Direction = GenerateReflectedRayDirection(Data.Ray.Direction, Payload.WorldNormal, Payload.Roughness, RandSample);
            ApplyPositionBias(ReflectionRay, Payload.WorldNormal, MaxNormalBias);

            FAlternativeData NextData = Data;
            NextData.Ray = ReflectionRay;

            // #dxr_todo: reflection IOR and clear coat also? This only handles default material.
            float NoV = saturate(dot(-Data.Ray.Direction, Payload.WorldNormal));
            const float3 ReflectionThroughput = EnvBRDF(Payload.SpecularColor, Payload.Roughness, NoV);
            NextData.PathThroughput *= ReflectionThroughput * vertexRadianceWeight;

            NextData.AlterTimes = Data.bReflection ? Data.AlterTimes : Data.AlterTimes + 1;
            NextData.RayTimes = Data.RayTimes + 1;
            NextData.bReflection = true;
            NextData.AlterSelect = 0;

            if (CheckNextDataValid(NextData, StackIdx))
            {
                Data.AlterSelect = 1;
                Stack[StackIdx-1] = Data; // Save Change
                Stack[StackIdx++] = NextData; // Push Stack
                continue;
            }

            //float3 ReflectionRadiance = AlternativeRayTrace(NextData, Params, NextAlterTimes, RayTimes + 1, true);    //cannot use recursion
            //PathRadiance += ReflectionRadiance;
        }

        // 5. Update the refraction path transmittance
        float PathVertexTransmittance = Payload.BlendingMode == RAY_TRACING_BLEND_MODE_ADDITIVE ? 1.0 : 1.0 - Payload.Opacity;
        Data.PathThroughput *= PathVertexTransmittance;

        //
        // 6. Handle refraction through the surface.
        //

        // Set refraction ray for next iteration
        if (Data.AlterSelect < 2 && TranslucencyRefraction)
        {
            FAlternativeData NextData = Data;

            float Ior1 = DielectricF0ToIor(DielectricSpecularToF0(Payload.Specular)); // Not using Payload.Ior but parameterisation from specular.
            float Ior2 = Data.LastIor;
            NextData.bHasScattered |= Ior1 != Ior2;
            bool bIsEntering = Payload.IsFrontFace();

            if (bIsEntering)
            {
                // Push Ior2 into stack
                if (NextData.IorStackIdx < MAX_IOR_STACK)
                    NextData.IorStack[NextData.IorStackIdx] = Ior2;
                NextData.IorStackIdx++;
            }
            else
            {
                // Pop stack
                NextData.IorStackIdx--;
                if (NextData.IorStackIdx < MAX_IOR_STACK && NextData.IorStackIdx >= 0)
                    Ior1 = NextData.IorStack[NextData.IorStackIdx];
            }

            float ReflectionRefractionEventThroughput;
            float3 RefractedDirection = Data.Ray.Direction;
            if (RefractRay(NextData.Ray.Direction, Payload.WorldNormal, Ior1, Ior2, bIsEntering, RefractedDirection, ReflectionRefractionEventThroughput))
            {
                NextData.LastIor = Ior1;
                NextData.Ray.Origin = HitPoint;
                NextData.Ray.TMin = 0.01;
                NextData.Ray.TMax = LocalMaxRayDistance;
                NextData.Ray.Direction = RefractedDirection;
                float SurfaceCurvature = 0.0f; /* #todo_dxr assume no curvature */
                NextData.RayCone = PropagateRayCone(NextData.RayCone, SurfaceCurvature, Params.Depth);

                NextData.PathThroughput *= ReflectionRefractionEventThroughput;

                NextData.AlterTimes = Data.bReflection ? Data.AlterTimes + 1 : Data.AlterTimes;
                NextData.RayTimes = Data.RayTimes + 1;
                NextData.bReflection = false;
                NextData.AlterSelect = 0;

                if (CheckNextDataValid(NextData, StackIdx))
                {
                    Data.AlterSelect = 2;
                    Stack[StackIdx-1] = Data; // Save Change
                    Stack[StackIdx++] = NextData;    // Push Stack
                    continue;
                }

                //float3 RefractionRadiance = AlternativeRayTrace(NextData, Params, NextAlterTimes, RayTimes + 1, false);    //cannot use recursion
                //PathRadiance += RefractionRadiance;
            }
        }

        // 7. Pop Stack
        if (!Data.bHasScattered && Data.AlterSelect == 0)
        {
            PathRadiance += Data.PathThroughput * Params.Scatter;
        }
        StackIdx--;
    }

    return PathRadiance;
}
#endif //USE_ALTERNATIVE_RENDER


  1. embed the function into RAY_TRACING_ENTRY_RAYGEN function (the entry of ray tracing in shader)


RAY_TRACING_ENTRY_RAYGEN(RayTracingPrimaryRaysRGS)
{
    // Original initialization codes
#if USE_ALTERNATIVE_RENDER
    float HitDistance = 0.0f; // Not use now

    FAlternativeData Data;
    Data.Ray = Ray;
    Data.RayCone = RayCone;
    Data.PathThroughput = 1.0;
    Data.LastRoughness = 0.0;
    Data.LastIor = 1.0f;
    Data.IorStackIdx = 0;
    Data.bHasScattered = bHasScattered;
    Data.AlterSelect = 0;
    Data.bReflection = false;
    Data.AlterTimes = 0;
    Data.RayTimes = 0;

    FAlternativeParameters Params;
    Params.bAllowSkySampling = bAllowSkySampling;
    Params.RandSequence = RandSequence;
    Params.bSkyLightAffectReflection = bSkyLightAffectReflection;
    Params.Depth = Depth;

    // Use the scene radiance for ray that has not been scattered/refracted (no surface or IORin=IORout). Still apply the throughtput in case we have traversed surfaces with opacity>0.
    Params.Scatter = SceneColorTexture.SampleLevel(GlobalPointClampedSampler, UV, 0).xyz / View.PreExposure;

    float3 PathRadiance = AlternativeRayTrace(Data, Params);
#else
    // Original codes, including for loop
#endif //USE_ALTERNATIVE_RENDER

    // Below is original codes
    if(ShouldUsePreExposure)
    {
        PathRadiance.rgb *= View.PreExposure;
    }

    PathRadiance = ClampToHalfFloatRange(PathRadiance);
    ColorOutput[DispatchThreadId] = float4(PathRadiance, 0.0f);
    RayHitDistanceOutput[DispatchThreadId] = HitDistance;
}


The pictures were at least triple added in the first posting. Creating a table with 1 column and # rows = # photos might solve that problem. Simply add one photo to every row. Or try attaching again. It’s really interesting that the initial thick glass refraction scales what’s behind it by default. Do you know how to explain some of these in simpler terms in addition to the math and technical aspects that were referenced?

Thanks your advice. There are a lot of introductions of the ray tracing mathematics, like WIKI(Ray tracing (graphics) - Wikipedia). And what I do is only simply fix some processes of ray tracing. About the convertion of IOR and Specular, there is a link describes it:

http://www.campi3d.com/External/Mari…ielectric.html

About refraction calculation, there is a link describes it well,

Hello, your fix is really awesome, since it finally let´s us reflect transparent objects/foils or masked objects, like clouds :slight_smile: (have not tested it with decals so far, but they should work too). however, i stumbled across some problem, when i wanted to reflect the transparent objects in solid metal objects (opaque objects set to metal = 1). I made a video, there you can see it, the transparent objects render black without color, but on closeup the color comes back.

I also noticed, that the new sunsky object, or its exponential height fog covers those transparent objects in the reflections of solid objects. The fog or atmosphere seem to be drawn on top of those transparent reflections and hiding them.

Thanks to point out the errors. I appologize that I have no response to you for a long time because I was in my hometown for spring festerval.
You are right that there are many problems when a metal object meets transparent objects.

About the problem that you relfect the transparent objects in solid metal objects, this is because the reflection of opaque objects in another function, RAY_TRACING_ENTRY_RAYGEN function in RayTracingReflections.usf. I didn’t modify it and actually we can use recurse method to make it done. In RayTracingReflections.usf, UE4 treats all transparent objects in reflection of metal as black objects. So it needs we to change more and the process of opaque objects is much more complex.

The fog and atmosphere can affect the opaque objects and its reflection but cannot affect the transparent objects.

Moreover, after you asked me, I found one error in my code (It’s actually error of UE4). In reflection of glass, the variable - bHasScattered - should be setted as TRUE as well.



        //
        // 4. Handle reflection tracing with a ray per vertex of the refraction path
        //

        // Shorten the rays on rougher surfaces between user-provided min and max ray lengths.
        // When a shortened ray misses the geometry, we fall back to local reflection capture sampling (similar to SSR).
        const float LocalMaxRayDistance = Params.bAllowSkySampling ? 1e27f : lerp(TranslucencyMaxRayDistance, TranslucencyMinRayDistance, Payload.Roughness);
        if (Data.AlterSelect < 1 && Payload.Roughness < TranslucencyMaxRoughness)
        {
            // Trace reflection ray 
            uint DummyVariable;
            float2 RandSample = RandomSequence_GenerateSample2D(Params.RandSequence, DummyVariable);

            RayDesc ReflectionRay;
            ReflectionRay.TMin = 0.01;
            ReflectionRay.TMax = LocalMaxRayDistance;
            ReflectionRay.Origin = HitPoint;
            ReflectionRay.Direction = GenerateReflectedRayDirection(Data.Ray.Direction, Payload.WorldNormal, Payload.Roughness, RandSample);
            ApplyPositionBias(ReflectionRay, Payload.WorldNormal, MaxNormalBias);

            FAlternativeData NextData = Data;
            NextData.Ray = ReflectionRay;

            // #dxr_todo: reflection IOR and clear coat also? This only handles default material.
            float NoV = saturate(dot(-Data.Ray.Direction, Payload.WorldNormal));
            const float3 ReflectionThroughput = EnvBRDF(Payload.SpecularColor, Payload.Roughness, NoV);
            NextData.PathThroughput *= ReflectionThroughput * vertexRadianceWeight;

            NextData.AlterTimes = Data.bReflection ? Data.AlterTimes : Data.AlterTimes + 1;
            NextData.RayTimes = Data.RayTimes + 1;
            NextData.bReflection = true;
            NextData.AlterSelect = 0;
            NextData.bHasScattered = true;    // In reflection, this variable also need to be true to prevent using pixels from SceneColorTexture.

            if (CheckNextDataValid(NextData, StackIdx))
            {
                Data.AlterSelect = 1;
                Stack[StackIdx-1] = Data; // Save Change
                Stack[StackIdx++] = NextData; // Push Stack
                continue;
            }

            //float3 ReflectionRadiance = AlternativeRayTrace(NextData, Params, NextAlterTimes, RayTimes + 1, true);    //cannot use recursion
            //PathRadiance += ReflectionRadiance;
        }


Don´t worry about that, festivals and family are more important than code ^.^ I am more than happy, that you try and fix raytracing and enable the still missing features :slight_smile: Take your time.

Do you know if this works in 4.25? Looks like they havn´t fixed the refraction working still. So I tried to set up the blueprints as you describe in the images just to see if I would get a refraction on a sphere at least. Didn´t go into the C++ code to fix it further. But I cannot make it work.

First of all I take it that Ray Tracing has to be enabled in project settings? Then I am curios about your material function IOR2Specular. In the formula “Specular = sqr((IOR-1)/(IOR+1))/0.08” there is a square root but you seems to have put in a power to exponent node instead? Further more I don´t quite understand what the scalar input values are. Looks like you have set them to X = 1 (RED)

Dividing by the power of 2 is the exact equivalent of square root. So it’s the same. I don’t know about those scalar inputs being x=1, which is red, but my guess is it has to do with converting the variable IOR to the variable Specular for use in UE4 ray tracing. If one was set to y=1, it would probably not work because of a change of variable. Both being x=1, red, denotes the whole function is converting the same variable, x, from one type to another, IOR to Specular.

Good afternoon!

Your edits are awesome.

Can you help me

I repeated all the steps in my project to achieve the same effect, but no visible changes occurred.

  1. I created an IOR2Specular material function
  2. I edited the glass material
  3. made changes to the shader (adjusted for version 4.25)

What am I missing?

I don’t think this fix is currently working with 4.25, Epic changed a lot stuff concerning translucency on this update,
for me the hard part was in updating this fix from 4.24 to 25, was that there is only one ior defined in the new translucency
and like you said, adding new ior1 ior2 doesn’t work in that case.
I couldn’t come up with a solution yet, hope maybe MikeZheng can give us some tips:))

Yes, the 4.25 changed a lot, and I haven’t touched this block of code for a long time. I hope my analysis above can inspire you, try your best to understand some physical principles, and try to modify it yourself.

Without ray tracing enabled for a dynamic-lighting only project, how would I define the IOR accurately with the fresnel function? Do I simply plug the value (e.g., 1.5 for glass) in the fresnel function node, or multiply it by the fresnel function output?

The docs usually lerp with fresnel as the alpha and IOR as the B (1 in A).

Ok, thanks. The non-RT translucency had a double reflection (with the second reflections being translucent too), and I tried to get rid of it by changing material details settings and succeeded. But now I don’t remember what setting I changed that did it, so I’m going to create a new material with the IOR added with fresnel since it wasn’t in the previous glass material I copied from a youtube video. I’m also getting flickering or quick-changing color brightness when up close to the current material on any object I’ve applied it to…any idea how to fix it or what’s causing it?

Share a screen shot of it?
also, which transparency mode and engine version?

Possibly the metallic value shift. Possibly.
try making it a fixed number to see if you get no flicker.
​​​Also if thats it, a bug report may be in order…

thanks, I’ll try that. I was questioning the lerp-ing of all those parameters, but got pre-occupied with it being a details panel setting and hadn’t thought to examine that in the graph.

link for the sript is not working, can some1 repost it?