Hi,
I have a quite complex problem. I’m trying to convert a glsl fragment shader to Unreal material nodes. The original shader isn’t amazingly complex but at its core it uses interpolation search and various iterations per pixel. I know that I could write the shader natively as explained here but honestly the documentation is quite lacking and it’s also more difficult to support multiple platforms. So I’ve decided to try the Material editor approach. During the convertion I have encountered the following problems:
To convert the complexity of the original shader in a readable “visual” form I’ve created several Material functions that are then called by the main material. Anyway the original 180 lines of code translated to several big material functions, but I think that this is an intrinsic peculiarity of the “visual nodes” approach.
Iteration isn’t supported in the material editor. Am I right? I’ve searched extensively but I haven’t found anyone asking the same question, something that appears strange to me since iterative code can be quite normal in shaders. To override the problem I’ve created functions that are called consecutively for the required number of steps and that have symmetrical input/output. So, for example, for a simple loop like this:
a = 0;
for (int i = 0; i < 5;i++)
a += function(a);
I’ve created a function that takes ‘a’ as input and outputs ‘a’. On the material there are 5 nodes of the function connected one after another. It works but it’s not very elegant, also it works only for very small cycles.
Now for the real problem. After finishing all the material functions I’ve put them together in the proper material. Unfortunately as soon as I connect the result to the output node the editor stops responding forcing me to kill the process. I’ve tried many times with the same result. I’ve then simplified the material a bit and finally I’ve got an error message:
Error [SM5] warning: Line number “32768” got beyond range
MaterialFloat2 Local31752 = frac(Local31738);
from …/…/…/Engine/Shaders/BasePassPixelShader.usf: 10: #include “Material.usf”
Uh oh, I think I know what’s happening here. I’ve opened the HLSL code window just to discover a huge shader made up of thousands of lines of code. The “not simplified” version of the material was probably so huge that the editor crashed before being able to report the error.
So basically 180 lines of glsl code are translated to >32768 lines of pseudo HLSL code, like these
The final question is: what am I doing wrong? Maybe I completely miss something and I’m trying to do something in the worst possible way. Does anyone have tried to do something similar? Am I the only one who have tried something so dumb?
Any help would be greately appreciated. Thank you.
thenk you for your reply. I can’t post the code directly since its owned by the company I work for but the (unrelated) original shader I’ve used as an inspiration is this.
Honestly I wouldn’t even bother doing this with nodes only. As you have noticed each basic operation is moved to its own line of code, so you quickly end up with hundreds or thousands of lines when doing something this complex. Especially with multiple loops and nested functions.
Try to write the functions to MaterialTemplate.usf (or a new .usf file) and then just call them in a custom node in the material editor.
it’s not that simple. In the last week I’ve tried different approaches and, honestly, none seems the right one. I’m quite disappointed by Epic’s lack of information on this matter both in terms of documentation and replies to users asking about this subject on the forum/question hub.
As I’ve said I’ve searched extensively online and all I’ve found are users left on their own… Basically there are three possible approaches to use custom shaders with UE:
Convert your shader in material editor format. This works only for simple shaders without loops or otherwise you’ll face my problem (editor crashes because of huge generated shader). It’s also unclear what kind of performances to expect since it seems that the “node to code” compiler isn’t very smart when facing “extreme” situations…
Write your own shader and vertex factory like suggested by the documentation. The problem is that you are left on your own. The implementation is insanely complex (at least if you don’t want to dirt your hands with low level engine code) and it’s obviously not guaranteed to work at all in future version of the engine. I’ve examined the few plugins that use this approach and honestly I didn’t like what I’ve seen. They “mess” with the engine renderer risking to break everything. VaOcean shader approach has been discontinued because of racing condition (in fact it crashes on 4.6 because it accesses render data during game update) while FluidSurface is overly complex (for what it does) and it seems to crash on certain hardware. After two days spent following this approach I must say that’s it’s definetely not the right way, for me. It’s almost better to write you own engine if you have to do that and having this kind of problems isn’t the reason why we are using Unreal Engine in the first place… This can be an interesting approach for someone who wants to learn how UE works and for personal satisfaction but it doesn’t suit well with a professional environment. I can’t live with the fear that my code will not work on certain hardware or that could have to be completely rewritten before a milestone because a new release of the engine changed something.
Use the approach suggested by . Honestly, I like this approach a lot. I think that this is sould be the only reasonable solution. Custom nodes in material editor are great, in theory, but practically they are almost useless. First of all custom nodes can have only one output. So if you shader calculates normals, positions and color data (for example) you are forced to split it in three functions, resulting in doing the same calculations three times. Then there’s the problem of multiple platforms support. I’ve tried to overcome this problem by writing the shader functions in an external usf file and calling an entry point function in the custom node, instead of pasting all the code into it. So, for example, the custom node expression is “return MyExternalFunction(input1,input2,input3);” and this should be cross platform compliant since it’s a valid instruction both in hlsl and glsl. But are the usf files cross compiled correctly? I still don’t know. There’s also another problem: every small change you made to an usf file in the engine directory will result in a massive shader recompilation. If the shader has an error and your recompile it within the editor (shift+control+.) the editor crashes abruptly. If you re-launch it, it crashes while loading, the only way to find the error is launching the editor in debug mode with visual studio and check the output window that reports the hlsl cross compiler output. Lastly you have to propagate your new usf files to every team member and remember to add them on every new engine release. One last thing: in their current implementation custom nodes seem to be more a “dirty hack” instead of a proper solution.
Recapping all the above: there are certain complex effects that can’t be done with the material editor alone. Is there a recommended approach or is it better to don’t even try?
An official position on this matter from Epic could help to shed some ligh on this topic.
Another one caught up in the complex web that is UE4 rendering.
You must understand that there are certain things that UE4 was NOT made for, such as tampering too much with the rendering stuff. I guess Epic saw no reason for game developers to use anything besides their material editor to do whatever. In a way, that’s kind of true, if your game isn’t overly complex.
Now, if you want to do more complex stuff, then the problems start, as you have obviously noticed.
I’ve been battling for some time to get an easy way to use compute shaders, for example, basing myself greatly in the plugins you specified.
Initially I also tried to check for a way to import GLSL code directly (as you would certainly like), but found no way. Odd thing is that I remember looking at the source code and seeing the engine can convert HLSL shader code to GLSL, so one could almost hope there would be a way to simply skip that conversion and provide pure GLSL.
Do you have shader development mode set to 1? That should cause the editor to prompt you indicating the error and asking if you want to recompile, without immediately crashing. If not, be sure to do it in ConsoleVariables.ini file in the Config folder of your engine (you just have to uncomment it, I believe).
For this kind of stuff I don’t think there is an official recommended approach. It’s almost every man for himself. Good luck!
Thank you dsl for your comment. Honestly I don’t completely agree with your statement about UE4 not being made for certain things. I think that the real problem is just lack of documentation (and support, in fact nobody from Epic replied) and that some parts, like custome nodes in the material editor, would require a bit more work from Epic. But unless you are making a very simple project there is almost certainly the need of a complex shader sooner or later. Most of those (but not all) can be made with the material editor alone but at what price? A few lines of code are often translated in incredibly complex visual materials that are a pain to mantain. Don’t get me wrong: I love the visual material editor, for someone like me that is a programmer for more than 20 years, writing shaders visually is almost relaxing, like playing a solitaire. But they are not very efficient in a production environment. I can write and debug a very complex shader in HLSL in a few hours while it could take days to obtain the same result with the material editor (just take a look to the materials contained in this project. I seriously respect the work of the guy who made that, but it can’t be considered more than an “academic exercise”. A Gerstner based ocean simulation is a quite trivial thing to do, while it’s taking MONTHS to the maker of that project. And not because he’s stupid, at the contrary. He’s just using the wrong tools…). So, in my opinion, this is something that should be addressed by Epic once and for all, even because they are so near to perfection that it’s a pity that they don’t make the last effort
By the way you were right about the shader development mode set to 1, I had set it in console but after a crash I’ve lost the setting and forgot to put it consolevariables.ini.
Anyway the good news is that I’ve been able to do everything I wanted (using custom nodes) and everything works like a charm, with no added complexity and remaining cross platform. It’s almost amazing how well it works, even if some parts are still a bit uncertain, that’s why I say that it doesn’t take much for Epic to finish the job.
@ernesernesto Later I will post a mini-guide on how I made it, I’m out of time now
I don’t have anything amazing to say, really. Everything I did has been done before, the only problem is that there’s not enough information around on how to do it properly. So here are my 2 cents about what I’ve found:
As stated on the official documentation and remembered by dsl is highly recommended to set r.ShaderDevelopmentMode to 1 before start working with shaders. By doing this you’ll get error messages when something goes wrong while compiling shaders and prevent the editor from crashing. Edit the “consolevariables.ini” file found in your (UEinstallationpath)/(version)/Engine/Config directory and uncomment “r.ShaderDevelopmentMode=1”
Forget the idea of converting your complex shader into material nodes. The custom material expression node is the way to go.
I suggest to test your vertex/fragment shader in a custom environment outside Unreal Engine first. You can write your own testbed in c++/DirectX/opengl or use Render Monkey or Fx Composer (they are discontinued but still working, more or less). This is because if you test your shader directly in Unreal Engine you’ll have to wait for a complete shader recompilation every change you make. This can take up to 5 minutes, depending on your hardware.
My primary concern was losing cross platform support. As stated on the official documentation “Shader code written in a custom node is compiled ‘as is’ for the target platform. This means that if the shader is being compiled for PC, it is assumed to be valid HLSL. If compiling for PS3, it is assumed to be valid Cg.”. A workaround I’ve found is the following: write your shader in a .usf file in the unreal engine shaders directory. Usf files are HLSL shaders that are cross compiled correctly on different platforms. Then use the custom expression just to call your entry function defined in the .usf. The call syntax is the same between HLSL, GLSL and CG so every platform will compile the call correctly.
From my experience custom expressions accept your custom code only if it is defined in MaterialTemplate.usf. I’ve tried to simply add my own .usf file to the shaders directory, it gets compiled but the functions contained in it aren’t accessible from custom nodes. I don’t like messing up too much with UE “system” files so to minimize the problem I’ve added a “MyShaders.usf” file to the shaders directory and then I’ve modified “materialtemplate.usf” to include my new file (#include “MyShaders.usf”). Using this method when a new version of UE is released you’ll just have to re-include your file in materialtemplate.usf.
HLSL code view (under window menu in the material editor) is your friend. From there you can see how your function is called and where it is placed in the rendering pipeline.
Unfortunately I haven’t found a method (I think there’s none) to pass data between shaders called in different stages in UE (you only have an output parameter in custom expression nodes, no way to send data to the stream output stage).
Finally a practical example. Let’s say that you want to create a material that generates a simple plasma on the applied mesh (you can do this without problems in the material editor, it’s just an example).
create the file “plasma.usf” and put it in your engine/shaders dir. The file can have the following content:
float color = 0.0;
color += sin(position.x - position.y) ;
color += sin(time)* cos(sin(time)position.yposition.xsin(position.x))+.008;
color += sin(time)+position.xsin(position.ysin(sin(tan(cos (time)))));
return float4( float3(sin(colorcolor)4.0, cos(colorcolor) , color )sin(time+position.x/(time3.14)),time/10.828 );
}
add "#include “plasma.usf” at the beginning of materialtemplate.usf
create a new material in your Unreal Engine project
connect the “base color” pin to a new custom expression node
add a total of three inputs to your custom expression node (uv -> type float2, time -> type float, phase -> type float)
define the output type as “CMOT Float 4”
in the “code” edit box write “return GeneratePlasma(uv,time,phase);”
connect the “uv” pin of your custom node to a TexCoord input expression
connect the “time” pin of your custom node to a Time input expression
connect the “phase” pin to a constant (ie. 2) or a scalar parameter.
compile the material, apply it to a mesh and that’s it
I know that there’s probably nothing new in my “guide” but I’ve lost several days on trial and error to collect all the information above. I hope that this post could help someone saving a bit of his/her precious time.
Thanks Gastel71 for your info; I’ll add a ticket for a doc write-up/blog post on how to add your own custom shaders. As you mentioned, it’s not straightforward due to the tight coupling between material editor, blueprints, lighting model we use, etc, but hopefully this doc can clear out some doubts.
As for writing glsl directly, the cross-compiler handles more than just translating hlsl->glsl (or Metal), it also generates reflection information we use to bind parameters & textures at runtime, plus it also puts global parameters into a packed array for less api setup (better perf). Writing hlsl off glsl is way easier than the other way around
Thanks RCaloca,
as you’ve noticed the biggest problem is the lack of documentation. I have an additional request though: do you think it could be possible for Epic to support custom shaders compilation located in the project directory? Similar to what happens with the “plugins” directory UE could check if the directory “shaders” is present in the project directory and compile the shaders contained within (exposing the compiled shader functions to material expression nodes).
Apologies for digging up more than a half-year old post, but I also wanted to say thanks to Gastel71 for his example on how to get a custom pixel shader working in UE4.
I’m working on a plugin which needs to make use of instanced rendering and requires both custom vertex and pixel shaders. I’m a total newbie to UE4, and information on doing any kind of custom rendering like this is pretty much nonexistent, so these little useful gems of knowledge are extremely helpful.
I was able to take Gastel71’s example and convert it straight to C++, with the added benefit of including the shader code directly within the source as a string and not needing to include a separate .usf file:
// create a package to hold our runtime created resources
UPackage *pkg = CreatePackage(nullptr, TEXT("/Game/HLSLTest"));
// create the custom expression with the HLSL code
UMaterialExpressionCustom *exp = NewObject<UMaterialExpressionCustom>(pkg, TEXT("Plasma_HLSL"));
exp->Code = TEXT("float2 position = ( uv * phase ); \
float color = 0.0; color += sin(position.x - position.y); \
color += sin(time)* cos(sin(time)*position.y*position.x*sin(position.x))+.008; \
color += sin(time)+position.x*sin(position.y*sin(sin(tan(cos (time))))); \
return float4( float3(sin(color*color)*4.0, cos(color*color) , color )*sin(time+position.x/(time*3.14)),time/10.828 );");
exp->OutputType = CMOT_Float4;
exp->Inputs.Empty(); // required: class initializes with one input by default
exp->Inputs.Add( { TEXT("uv") } );
exp->Inputs.Add( { TEXT("time") } );
exp->Inputs.Add( { TEXT("phase") } );
// create and link input expressions to the custom expression node
UMaterialExpressionTextureCoordinate *uv = NewObject<UMaterialExpressionTextureCoordinate>(pkg, TEXT("Plasma_UV"));
exp->GetInput(0)->Expression = uv;
// using UMaterialExpressionTime results in a linker error?
// UMaterialExpressionTime *time = NewObject<UMaterialExpressionTime>(pkg, TEXT("HLSL_Time"));
UMaterialExpressionConstant *time = NewObject<UMaterialExpressionConstant>(pkg, TEXT("Plasma_Time"));
time->R = 2.0f;
exp->GetInput(1)->Expression = time;
UMaterialExpressionConstant *phase = NewObject<UMaterialExpressionConstant>(pkg, TEXT("Plasma_Phase"));
phase->R = 2.0f;
exp->GetInput(2)->Expression = phase;
// create a material to be applied to a mesh (must be public to allow for serializing!)
UMaterial *mat = NewObject<UMaterial>(pkg, TEXT("Plasma_Material"), RF_Public);
mat->Expressions.Add(exp);
mat->BaseColor.Expression = exp;
This then gives a UMaterial which uses the custom HLSL pixel shader code that can then be assigned to a UStaticMesh with SetMaterial().
I couldn’t get UMaterialExpressionTime to work due to a linker error, so unfortunately the plasma doesn’t animate. I’m still on UE4.8, so this might have already been fixed (4.8.3 was just released today).
The next step from here for me is to figure out how to use a custom vertex shader. I’ve already looked at VaOcean and FVertexFactory, and am hoping I don’t have to go down that route.
