Extending Custom HLSL / Custom Expressions

Until we see some more love for fast, custom HLSL in Unreal, here is a discussion of my recently discovered tricks to go beyond the current limits.

#1 - Custom Functions

Modifying the Engine’s USF files to define custom functions triggers an overhaul shader recompilation of thousands of materials. This is unusable for creative iteration. Also #include-ing files outsides the Engine’s Shaders directory crashes the Editor at startup. CustomExpression nodes wrap your code inside CustomExpression#() functions, and that normally prohibits defining your own functions.

However, there seems to be a barely documented feature in HLSL that allows defining functions (methods) inside struct definitions. struct definitions can be nested inside functions (like the wrapper CustomExpression#).

So in your CustomExpression Code you can do:



struct Functions
{

  float3 OrangeBright(float3 c)
  {
      return c * float3(1, .7, 0);
  }

  float3 Out()
  {
    return OrangeBright(InColor);
  }

};

Functions f;
return f.Out();


Create and connect an input pin “InColor” of float3 on the CustomExpression node. Any Node inputs passed into CustomExpression#(), like InColor above is available inside nested function definitions.

The cool part is, this is all happening inside your own effective namespace, not interfering with Unreal’s USF files, and the compilation is Fast for iteration. So now, you can start to build a library of custom functions, and more complex shaders. It seems HLSL is prohibiting defining a struct nested inside a struct, so make sure to define your custom structs above and outside struct Functions.

#2 - External Code Editing and #include

Instead of editing intricate code and custom libraries inside the little primitive textbox of CustomExpression, you can edit them in a better external editor with syntax highlighting, code navigation etc, and #include that file. So if you put the above code in a file named Test.hlsl, you can:



#include "Your Path...\Test.hlsl"
return 0;
// enter spaces here and press enter to retrigger compilation


The dummy “return 0;” is to tell CustomExpression node that this not a single line expression but a full function body. The spaces will be required to signal the CustomExpression textbox that it changed, and pressing enter will compile your externally changed and saved Test.hlsl. Of course, you can split the external file and the dynamic code portion, if you prefer to make quick changes and compiles inside the textbox.

#3 - Multiple Outputs

The CustomExpression node limits the output to a float4. I have some ideas using CustomExpression#() chains and injecting macros to redefine their definitions and calls later to create additional outputs or complete Material outputs later at CalcPixelMaterialInputs() etc. But let’s see how the above tricks work for people, before we delve into that.

#4 - Multiple Passes

Some image processes like convolution are far more efficient when broken into multiple passes by storing intermediate results in a texture. Currently, if you have a process that comes before convolution, a previous function has to be called redundantly for all the neighborhood kernel samples. There is some hope in Unreal’s render targets, but whether they can render multiple passes per frame synced, etc, and how much more spaghetti they will create with additional pass materials and control blueprints, we should discuss… Ideally, texture writing should be available completely inside the Material Editor, there should be variables, true branching, and custom #includes at the global scope, all features requested for years to take the already excellent default PBR system in Unreal to the next level with industry leading oceans, volumetric clouds, professional chroma key, etc.

5 Likes

Branching inside editor and more outputs from custom node had been requested for quite some time indeed. Though personally, I would prefer a convenient code shader authoring system side by side with material editor.

these are some great findings. the Multiple Outputs is something I had been needing recently so it’d be great if you could shed some light on it :slight_smile:

I’ve given MultiOut more thought this morning, and ran some tests. I have a design that would be quite easy to use with one #include for your custom program, another #include with the preprocessor magic. You would set the path of where you keep your includes in the first CustomExpression node, and chain additional CustomExpression nodes for additional float outputs. Your program will run first, outputting an array of floats of a given count, and the additional CustomExpression nodes in the chain will output individual floats from that output array. Only limitation in this easy-to-use design is that you would only have one MultiOut program per material.

Great finds Büke! I was wondering, what if I’d like to use a relative path to the external .hlsl file? Do we know which folder it would be relative to?

Got it: It’s the ‘Engine\Shaders’ folder.

Hello, Buke could you please give a more detailed example as how you realize MultiOut? I still don’t quite understand. Thanks a lot.

Hey,
is this still working?
I am trying to import some custom files as described:

placing the .hlsl file in Engine\Shaders and using relative path, but I am getting this issue:
if I name it .hlsl I get:

[SM5] (): Extension on virtual shader source file name “/Engine/Generated/test_struct.hlsl” is wrong. Only .usf or .ush allowed.

I try to rename it:

[SM5] /Engine/Generated/Material.ush(1428): error: Can’t open include file “test_struct.ush” include “test_struct.ush” from /Engine/Private/BasePassVertexCommon.ush: 9: include “/Engine/Generated/Material.ush” from /Engine/Private/BasePassVertexShader.usf: 7: include “BasePassVertexCommon.ush”
[SM5] /Engine/Generated/Material.ush(1428): error: Can’t open include file “test_struct.ush” include “test_struct.ush” from /Engine/Private/BasePassPixelShader.usf: 9: include “/Engine/Generated/Material.ush”
[SM5] /Engine/Generated/Material.ush(1428): error: Can’t open include file “test_struct.ush” include “test_struct.ush” from /Engine/Private/ShadowDepthVertexShader.usf: 12: include “/Engine/Generated/Material.ush”
[SM5] /Engine/Generated/Material.ush(1428): error: Can’t open include file “test_struct.ush” include “test_struct.ush” from /Engine/Private/ShadowDepthPixelShader.usf: 12: include “/Engine/Generated/Material.ush”

Any suggestions? Thanks :slight_smile:

Try throwing a:

#pragma once

at the top of your shader file. I make all sorts of custom shaders like these and use the include / return 0 custom node method for them in 4.19.2. If that doesn’t work, then check your shader code. Make sure you’re using the struct setup properly.


#pragma once

struct TestingMath
{
float TestFunction(float Ain, float Bin)
{
return Ain + Bin;
}
};

TestingMath TestName;
return TestName.TestFunction(A, B); //pin input names

Saved this file as TestMath.ush within the shaders/private/ folder. The include will be Engine/Private/TestMath.ush and don’t ask me why the directories don’t line up with the actual real folder structure. I just know it works this way…

Awesome, it works thanks!!

It would still be awesome if we could just add the include in one custom node and use the functions in the others like this…

Unfortunately it doesn’t work!

Code of the structure it’s just:


#pragma once
struct IacopoHlslFunctions
{
    float3 OrangeBright()
    {
        return float3(1, .7, 0);
    }
};    

IacopoHlslFunctions iacopoHlslFunctions;

I would expect the iacopoHlslFunctions variable to be added to the scope of the material, apparently it’s not.

This other code works fine, but every time you use a custom node (in the same material) you need to re-include the file, which I don’t understand why.


#include "/Engine/Private/iacopoHlslStructures.ush"
return iacopoHlslFunctions.OrangeBright();

I want to say that you can do that if the function is in the far left, and is linked into another custom node. It doesn’t have to feed any information into the second custom node, you just need a pin running into a dummy slot so that it gets linked during compilation.

Awesome! Thanks for your sharing. @IronicParadox @****Büke

Heya @****Büke, can you help me understand why I get


[SM5] /Engine/Private/test.usf(13:25): error: reference to local variable 'InColor' declared in enclosing function 'CustomExpression0'
        return OrangeBright(InColor);
                            ^


. I’ve spent hours trying to understand. It just seem like I can’t pass a node pin variable into a function that lives inside a struct, but somehow everyone here compiled. I don’t know what i’m missing.

full code is


#pragma once
struct Functions
{
  float3 OrangeBright(float3 c)
  {
      return c * float3(1, .7, 0);
  }
  float3 Out()
  {
    return OrangeBright(InColor);
  }
};
Functions f;
return f.Out();

I do have one workaround which is to pass the node pin variables in as parameters at the final return outside of the struct. But this workaround is useless : / So for example:


float3 Out(A)
{
return OrangeBright(A);
}

...

return f.Out(InColor);


Hiya, sorry for the zombie resurrection but I’m getting this reference to local variable error issue as well (in 5.2). Are there any [better] work-arounds that have been discovered yet?

This is how i solved this problem:Each struct contains one function

struct hash
{
float m_hash( float2 p ) {
float h = dot(p,float2(127.1,311.7));
return frac(sin(h)*43758.5453123);
}
};

struct noise
{
hash mhash;
float m_noise( float2 p ) {
float2 i = floor( p );
float2 f = frac( p );
float2 u = ff(3.0-2.0f);
return -1.0+2.0
lerp( lerp( mhash.m_hash( i + float2(0.0,0.0) ),
mhash.m_hash( i + float2(1.0,0.0) ), u.x),
lerp( mhash.m_hash( i + float2(0.0,1.0) ),
mhash.m_hash( i + float2(1.0,1.0) ), u.x), u.y);
}
};
noise mnoise;
return mnoise.m_noise(iTime);