Writing custom HLSL shaders?

I’ve been looking into rendering realtime fractals in unreal engine for VR.

Obviously this would be quite expensive, so doing a bit of digging I’ve found that people have been able to write custom shaders in unity to render signed distance functions.

Process: https://.github.io/vr/fractals/2017/12/30/RTA.html

Another Article about rendering SDF’s: Inigo Quilez :: computer graphics, mathematics, shaders, fractals, demoscene and more

I’m new to HLSL and learning slowly, however all tutorials and information are abstract to my use case, and I can’t seem to find any good guides on writing custom shaders for Ue4

Before you start doing any custom HLSL code in UE4, make sure you really need to do that. Materials are shaders, the way it works is whole graph is converted to HLSL (you can even see whats graph generating, there option in Window menu), because of that there no performance hit because of using Materials as if you do same calculations with will be as much as expensive as it would be in HLSL directly in fact UE4 will try to optimize it more. So if you can try to port calculations you trying to get in UE4 using material nodes, i myself had successful attempts doing so. Remember that you can do material functions, as lot of HLSL code can extensively use functions. Only downside of using materials, is that you are limited to nodes available, some things are not controllable and there lack of looping support, then indeed using HLSL coding might be better solution.

Now there 2 ways to use HLSL direcly in UE4, most simplest one and less invasive is custom code node in material:

It works like a function, which you input arguments via input pins and return value becomes output pin. You still limited on what UE4 provides for and do to material, but you may use some things not avable in nodes and you can do loops.

2nd is writing custom HLSL shader using usf files, issue is they are not very well documented (because Epic expect you to use materials for most things) and usually requires C++ coding:

If you google usf you might find a lot more stuff.

One of biggest challage of making custom HLSL code using usf files is that you placing code in to already complex shader system that drives all effects in UE4 and you code need to follow the conventions established by it so you don’t have any conflicts with it.

Unity indeed deals with it a lot better, but on other hand they use more direct HLSL because doing material system that UE4 have is quite a feat.

1 Like

The problem with Unreal is that it defaults to these editor-level tools as the proposed “primary solution” instead of having them as a “secondary” one.

There’s a common pattern emerging through my research on anything about this engine that isn’t already provided to you and it’s that the engine was designed in a way that makes it multiple times more hassle and bother to work with anything in this engine that doesn’t already support what you’re trying to do? Quite the opposite of productive and ideal, I’d say.