Is Epic doing anything to address UE4's crippling shader problem?

(Sorry for exaggerated thread title. You’re doing a good job Epic :cool:)

Ever since UE4 was released over two years ago, people have been complaining about the shader extensibility problem. The material editor is very limited and should never have completely replaced HLSL shader coding like it does now. There are too many things it can’t do:

It can’t do new shading models that react to UE4’s lights
It can’t do geometry shaders
It can’t do multi-pass shaders
It can’t do compute shaders
etc, etc…

These are serious problems for anyone who is interested in getting creative with rendering or giving their game its own unique look.

Can we take this opportunity to discuss this issue? Is anything being done to solve this and allow users to simply create new .usf shaders files in-editor? If not, can you share details as to why, or what that would involve? Is there no way to make a material type where you directly write HLSL vertex/geom/pixel shaders and have it generate code into the engine’s various .usf files at all the right places when you press Apply? Or something along those lines…


1 Like

I’m no shader expert, but does the custom node help?

the custom node seemed like a light of hope at first, but it’s really not much more than a place where you can just write math/loops more easily. Its limitations are explained at around 5:30 of this training video: Custom Material Node: How to use and create Metaballs | Live Training | Unreal Engine - YouTube

It doesn’t actually let you create shaders or access things like lighting info. It basically only lets you transform inputs that are fed into existing shaders

Material editor is not right place to do everything. In deferred shading shading is strictly separated from materials. Things like “lighting info” isn’t just there when draw call is issued.

Yes, absolutely. But the problem is that UE4 doesn’t provide a system that allows users to write new shaders, and “just use the material editor” is often the answer I get when I ask about how to write shaders for UE4.

Seems like the material editor was intended as a complete replacement to HLSL, so that “people would never have to write scary shaders again”, and now we are very limited by that design choice

I’ve never checked as I am no expert with shaders, but is there really no place to drop custom HLSL code? If so, that seems like a massive oversight.

The only way to develop material shaders currently is by modifying the engine source. I’ve actually done the procedure not too long ago to create a toon shader, and it goes something like this:

  • You have to add a definition for your new shading model in 3-4 engine source files
  • You have to add definitions for your shader inputs/pins in 4-5 engine source files
  • You have to recompile the entire engine
    Result: you now have extended the material editor to add a new shading model choice alongside “Unlit, DefaultLit, Subsurface, etc…”

Now to implement the shader, you have to modify 6-7 different .usf files in the engine’s ‘Shaders’ folder, to integrate your new shading model into the defferred renderer.

Once you’re done writing your shader across all of these files, you open up your custom engine build and press ‘Ctrl+Shift+.’ to recompile all 5000+ shaders. This takes about 20 minutes.

Now you have successfully created a new material shader. And every time you make a change in the shader code, you have to recompile all shaders and wait 20 minutes again. Also, you’re stuck with a custom engine build, so you have some merging work to do every time you wish to update.

Needless to say, this is really really bad for iteration. It takes 6 hours to add a shader and make it work, and another 20 mins everytime you want to change it. Doing things this way isn’t really a solution.

It’s actually quite easy to write your own shaders (outside of the editor) and to use them in UE, but in general these shaders cannot be used together with the material editor or to replace the default shading model in UE. I’ve used custom shaders mostly for compute tasks and for some custom rendering to texture effects (where you need to manually specify all inputs, etc…).

One exception are custom vertex shaders that can, at least to some degree, interact with the material editor if they are written using the vertexfactory framework…

Just curious; is it necessary to recompile all shaders everytime you make a change when you do this?

And I’ve never really found any helpful documentation on working with compute shaders in UE4. Could you give us any tips? Any sort of compute shader ‘Hello world’ would be a huge help

No, as far as I remember, only the compute shaders are recomputed, the same is true for other isolated shaders that are not linked to material editor.

As for documentation, I started working on it following this tutorial: [Tutorial] Pixel and Compute Shaders in UE4 - C++ Gameplay Programming - Unreal Engine Forums
It contains a source that may be somehow outdated but it should still hopefully work. Saying that, a good documentation is still lacking, especially when it comes to setting shader input and output parameters that can come in many different forms. I often ended up digging quite deep in the source to figure out what classes to use for the input parameters I needed.

Is there something in particular you want to do? Epic has an official Parallax Occlusion method in the material editor. You can use unlit mode and light vectors/reflections/forward shading with blueprints, if you want. The only thing the material editor can’t do is geometry shader stuff like fin and shells, but nowadays the fin and shell technique is both outdated and too expensive for today’s games demanding higher resolutions and more complex specular reflections/deferred shading. There are some other hair solutions you can try first before giving up on the material editor completely. It’s very easy to use, and very easy to make complex surfaces.

Yeah I think the need to modify the engine source is really inconvenient… it should at the very least be possible with a plugin and not have to manually copy .usf file to the engine folder but read them directly from the plugin directory.

There are several things on my mind:

1- A toon shader with outline (not as a post-process, because it doesn’t give as great of a result). So far, I’ve managed to go through several days of pure suffering in order to create an entirely new shading model that I can choose in the material editor dropdown. This is the result. But I’m still missing the inverted hull toon outline that would require a geometry shader for my toon shading model. I have yet to figure out how to make geom shaders work in the pipeline.

2- Something like this. This should be much more straightforward than creating a new material shading model, but still, documentation of creating shaders is so absent that I fully expect this to be a living hell too. At least ostava’s link above might be helpful

3- Compute shaders. GPU-accelerated pathfinding, GPU-accelerated mesh deformations, GPU-accelerated anything, really

But, forget about the 3 things I just mentioned and just remember this: I want to know that I have the ability to deal with any shader-related situation that might come up down the road. I do not want to start a project knowing that I will have to rely on Epic giving me pre-made solutions to all of the individual problems that I could encounter later. I want to have the freedom to experiment with shaders. This is by far the biggest issue for me.

Epic, would it be possible to just ask one of your rendering people to take a few days to write complete and thorough tutorials on creating:

  • A new material shading model (just some ultra-basic phong shader with a new input property, for example)
  • Regular shaders (vertex, geometry, and pixel shaders. Like a basic particle shader that draws a sprite on a given vertex point, for example)
  • Compute shaders (Make a basic GPU particle emitter using the particle shader we created above, for example)

It would be a huge help, and I would be eternally grateful to you. Even if we still have to modify source code.

1 - I know some things are not possible now that the engine switched to deferred, like anisotropic rendering. It’s a bit of a give and take: do you want custom shading, or the ability to render thousands of dynamic lights on the same mesh? I definitely wish there was a method to enable some forms of custom shading, though, like cel shading to work with deferred lights. You can always use the light vector method to feed into emissive, but that is a bit of a cop-out. At the moment, the stylized demo is your best bet. If you want UE4 to work for you, you need to find a method that the engine likes. It doesn’t really like translucency, and it doesn’t like crazy non-realistic rendering that isn’t at least somewhat based on the real world. If you want to make your own shader suite to be compatible with the material editor, good luck.

2 - The example you showed me looks like something that can be done with GPU particles in Cascade… unfortunately, there won’t be specular highlights unless you turn on forward rendering, and doing that will be very expensive because of the GGX. A lot of these tech demos are really awesome, but integrating it into a game might not be practical because of the world of light types and rendering features. If you use Epic’s systems, you will be amazed. I got caustics underwater from a light function, and the caustics would actually blur and light up beneath the subsurface material on my character and shadow itself between blades of underwater grass.

3 - There are some predetermined compute shaders like tessellation available for everyone… to make one yourself though, yeah, I don’t think that’s possible.

I do believe I have made the suggestion to support other models of specularity and have access to other functions, but by and large unless you’re using C++ to code a game, rendering stuff should not need to be coded. There is a UE4 solution for practically everything, except some of the more extreme cases. Unless you specifically want to program compute shaders on the GPU, it’s really not necessary. You can use GPU sprites in Cascade. You can use Parallax Occlusion Mapping within the material editor. You can’t do cel shading specifically, but if you want something like the stylized demo, you can totally do that. You can make a high-end cartoon style like Ratchet and Clank fairly easily in UE4. If all you want to do are things that UE4 can’t do, then, maybe it’s not the engine for you. But if you take advantage of the tools you have, you’ll be surprised what you can make.

I can’t agree with that kind of thinking. Good shader extensibility in UE4 is something that is within reach if some efforts are made to enable it, and that’s why I try to push things a little. Many people know how to program shaders, and for those people UE4 feels like a cage right now. There is just so much interesting potential you can unlock with shaders. There’s no reason not to want them.

Besides, I already said those three cases were just examples, and what I’m really interested in is knowing I won’t hit a roadblock just because the engine doesn’t enable me to come up with my own solutions. Technology is part of the craft of making games. It would be sad if all we could build our games with were pre-made building blocks that everyone else uses. Sure, you have to be realistic about it, and it’s probably not worth it to make an entire engine from scratch by yourself instead of using UE4… but making shaders IS realistic.

It may strike you as something that will only benefit an extremely small amount of people at first, but I don’t think that’s true. The entire community could benefit from shaders becoming available as plugins, either on the marketplace or for free. For instance, there is very clear demand for non-physical shading, and there’s nothing UE4 “doesn’t like” about it. It works perfectly well. It’s just that UE4 makes it a huge pain to add to the engine, and most people won’t feel comfortable moving to a custom branch of the engine.

I also would like to add new shading models with material editor. But this still would need to recompile all materials and lighting shaders. There are also hard limit how many shading models can be supported but this could be relaxed by making all shading models optional which would help with performance if you don’t use all of those shading models. I personally only use default, unlit and subsurface profile.

I’m not sure if the shading model needs to be determined by the g-buffer, I just thought that each object has a draw call for the material. I do know that there is a limit on g-buffer inputs like Color, Normals, Opacity/Metallic, Subsurface Color, etc, and using larger precision values for the g-buffer will hinder performance, but for the most part I think if you’re willing to sacrifice the extreme precision of 64-bit normals and colors, you can potentially gain enough of a performance benefit to make 8-bit masks for new models and 24-bit color inputs. If Epic has this kind of scalability, I could foresee people sacrificing quality for new shading models which would be beneficial for anyone wishing to render a combination of toon shaders with more complex ones. But the problem with a lot of tech demos is they never integrate practically with fully-featured engines. Maybe you CAN render nice water with caustics lighting and interactivity, and there are some fantastic Unity Web GL demos showcasing this is possible, but just because it works well on a 4-panel cube doesn’t mean it will work well over a lake with landscape and have enough performance benefit to run properly. We’ve seen tech demos of interactive water and hair, but I have yet to see a game seriously pull it off well. Cloth is still an issue for some reason.

“UE4’s crippling shader problem”?? :eek:
UE4’s shading system is so ridiculously good it’s crazy, it’s difficult to return to mortal software after using any UE4 features. Obviously you seem to disagree, but surely “crippling” and “very limited” is excessive, don’t you think?

I don’t want to devalue your findings however, since you seem to have reached the cap of what’s possible to do with UE4’s shading system you must obviously know an enormous deal about shading. But what if we ask this question instead, how many others are in need of the features that you ask for? The other day I was really disappointed there was no way to bake lighting into vertex lighting, which in my particular case would have been advantageous, but I just thought to myself that UE4 is after all not written specifically for my current projects needs, so I will trust that the devs probably made a just decision not to prioritize it based on what they know about their audience.

This is a point where Unity is way easier to deal with.

My first impression when I launched material editor was overwhelmingly positive. Easy to use, fast to prototype. However as I moved further, I’ve noticed that a good deal of tasks could be done more efficiently by just coding. No so long ago I discovered another unpleasant surprise. You cannot benefit from using dynamic branching with material networks. I’ve covered the issue in this thread](Dynamic flow control in materials - Rendering - Unreal Engine Forums).

Custom node solves some of the issues, but only to very limited extent. Personally, I would prefer an ability to output more data from custom node, rather than float4 only. That would suffice for me and the project I am into.

Adding to that and as stated by OP, there are quite a few things that are trivial to make with forward rendering path and multi-pass shaders, but doing it in UE4 turns into a challenge.

However, that is just my view, according to the tasks I am doing now. I’m quite sure that having access to coding shaders is far not the most requested feature, simply because of the fact that team, that needs features like that, usually can afford to modify the engine as needed. Less advanced projects simply never bump into this task.