Is Epic doing anything to address UE4's crippling shader problem?

Here’s another programmer unhappy with the current shader handling. It’s a pity, one of the best things invented for graphics programmers was a nearly cross-platform programming language (HSL, GSL). A clean structured and language to express complex operations with minimal code. Look at shadertoy.com to get an impression, what people can do, when they get immediate access to the graphics (haha, and sound :))
The OP has a good point, we want fast iteration, while developing graphics solution, a code change should not need minutes to compile.
And we don’t want to suddenly hit a showstopper just because the engine has artificial limits.

Having said that, Unreal4 has an amazing renderer, but needs to be extendable with plugins.

Agreed. Graphics programmers should be able to do what they’ve been trained to do - write HLSL/GLSL shaders with all sorts of crazy techniques, not just throw down nodes. It’d be like forcing C++ programmers to always use blueprints instead of making them optional #FreeTheGraphics2016 :smiley:

Just a quick example of a flaw in material editor. Each time a Texture Sample material node with a normal map is used, a function “UnpackNormalMap” from Common.usf is called. It seems to be used for every normal map sampler node, disregarding what operations are conducted further down the road. UnpackNormalMap function is pretty simple, just a few instructions, however when we are speaking about using a good deal of samplers and lerping between sampled values while unpacking normals in the end, savings become considerable. For 3 samplers lerped together,implemented in a custom node, as opposed to material network, as much as 10 instructions can be saved. I have firm opinion, that whoever authors the material, should have control over where exactly he wants to unpack the normal map.

This is just one small piece on the surface of the problem.

I’d argue that the Material Editor isn’t really designed for super-advanced things.

Just being able to add shaders without needed to work from source would be a major benefit though. Supporting those via Plugins or dropping them into a project somewhere would be superb.

I wonder how complex of a modification it would be to have the engine look in PROJECT/Shaders for .usf files aside from just Engine/Shaders… Would be a start.

Hey, thanks for the feedback on Unreal’s shader features (or lack thereof). I’m keeping an eye on all the suggestions and requests in here, so please keep them coming.

Why do you think they did that?
Try turning on the “deferred / G buffer” display, and think about how “objects,” “surfaces,” and “lights” interact in the current deferred renderer.
Now, think about what kinds of math needs to go where in this model.

If you want to shade an object differently, in a way that can’t be calculated as separate “surface properties” plus “light function” passes, then you can’t use the current deferred renderer setup.
You can, however, do all the calculations you want in a single shader, and then output that to the “emissive” property of the object, and output black/0 to all the other slots.

If you want a forward renderer to plug into, perhaps you can take a look at the mobile renderer, and see if that can be adapted to desktop use.
That’s been a request for quite some time.

Post above me quoted this and I had to respond.

I strongly disagree with this. The material editor is one of the best tools in the engine and a major pulling feature. It’s a tool for artists and should be presented as such so that the artists get the benefit from it. The material editor is one of the reasons why games coming out of Unreal look so much better than the competition - and there’s a reason why the competition tries so hard to mimic it.

Graphics Programming (HLSL / OpenGL etc) and Material Creation are two very different kettles of fish done by people with very different talents. All of the things you’ve listed can be done, compute shaders etc - but they require you to be working with .usf files and HLSL, and both of those things also require a bit of learning of the API too.


All this aside, I still agree that Graphics Programmers should have more room to play when it comes to things like this. A separate lower-level shader editor, or at least the ability to pull shaders in from outside the engine directory would be a major benefactor.

Of course. I only meant that I don’t like how the material editor is the ***only ***real option for messing around with shaders/materials, and that no real effort was made to make a good workflow for writing shader code

Yeah I understand, agreed on that one.

Node based shader editor is an amazing tool, no doubt, yet it is not in any way unique to UE. While material editor ensures that complex visual effects are achievable without diving lower into tech details, it also takes part of responsibility for low performance and feature restrictions. This is where it is falling back behind competitors you are mentioning. I bet most of the people who post in this thread belong to “I want more advanced shader stuff, but I cannot/do not want to/prefer not to make changes to the engine”. I cannot guess what percentage of users they constitute, but for them material editor is like a cage. It should not be interpreted as whining, that something is bad, but rather as attempts to make something, that is good, even better.

I must admit, so far I haven’t come across a situation, where I would unavoidably need to code a shader from scratch for UE4. Yet, quite often I see situations where part of a shader(pixel shader mostly), could have been done faster and better in a code. There is a custom node already, but it is missing a good deal functionality, sadly. A couple of times I would also have prefered to edit shadow pass shader separately.

In my view, one of these actions would mostly cover vacuum in the area of shader development in UE4:

    • an option to edit generated code, coupled with an ability to give sensible names to the variables.
    • ability to create new shader from template, and manually populate PixelMaterialInputs.
    • expansion of custom node functionality. Something, that would allow to either output several float4’s or an ability to write to PixelMaterialInputs from the custom node.

Like I said that’s all fine, but those kind of things should remain separate to the existing material editor. Graphics Programmers should have the freedom you described, but it shouldn’t be mixed in with the artists’ tools so that they are also forced to use it.

The Material Editor IMO should remain simple to use, over-complicating it would just give us a different kind of problem.

Last time I used the shaders yes the engine recompile all at restart.

Anyway I see this thread as well Blueprints are limited too, but what limit you of edit or create new shader files in the core of the engine ? The engine come with all the sources.

Yes it is doable, but it takes so much time and is so undocumented that it doesn’t seem like a viable option. And then you’re stuck with a custom branch of the engine that you have to manually update yourself.

Creating a new shader should be as simple as RightClick > Create Shading Model. You’d have an easy way to define what the inputs of that shading model are (either in code or through the editor), and then you’d write your shader code, and finally you could select it in the material editor to build a material based on that shader

Bonus points if there is a way to not recompile all shaders for 15 minutes every time you make a change

Suggestions for , and examples for anyone dismissing the need for custom shader code injection:

You’ve got an HLSL DX11+ deferred renderer with UE4. This means you should be able to:

  • implement raytraced geometry by writing to the color, depth, normals etc GBuffers before unreal’s lighting stage
  • do custom fluid simulation with better / custom techniques, distance fields, 0 meshes, and custom velocity grids
  • implement a raytraced CSG distance field sculpter for VR
  • do custom distance field physics and lighting
  • raymarch geometry or distance fields into voxels
  • bake custom distance fields into textures and access them in screenspace for correct flow simulation
  • implement the Dreams game from MediaMolecule or any SIGGRAPH paper
  • implement multi coloured outlines - which did and was a pain: forums.unrealengine.com/…/Odessey-Creating-my-own-G-Buffer-in-UE4
  • directly manipulate buffers (e.g. RWStructuredBuffer) and 3D textures, implement custom data structures entirely on the GPU (e.g. Octress), create new buffers and access them at multiple stages of the pipeline.
  • many things you haven’t dreamt yet you could do in UE4.

You know how people complain all UE games look like UE games? That’s because they all use the material editor and the same post FX.

All graphics and tools programmers need this.
Unless you only live under the UE rock, you know most tech demos for new and custom render methods are created in Unity. Unity knows this and is putting a lot of effort into making their Command Buffer even better and more flexible / extendable. You (Epic) will inevitably notice this is eating part of your pie, and start to fix things, but I/we just want to move this along faster than that because we already know of these problems.

PS: What’s the deal with UE4’s temporal screenspace ambient occlusion? It still looks like UDK3’s, and it’s nearly 2017… If you make the API more friendly, I’ll gladly fix it for you :wink:

PPS: To reiterate what’s already been said: it’s the wrong thinking to say “we’ll just add every conceivable or demanded feature to our material editor instead of making the code API more extendable / practical”. It’s as stupid as Unity thinking “our users should never need access to our C++ source”.

What do you mean? You think SSAO is not looking good in UE4? Could you show us a comparison where SSAO looks better than in UE4?

who would dismiss that? I mean like a whole thread is about need for more easily accessible shader authoring pipeline.

This https://forum.unity3d.com/threads/lightning-fast-quality-occlusion-simple-ssao.388820/,
id tech 6: http://www.adriancourreges.com//2016/09/09/doom-2016-graphics-study/,
and the option of coloured ambient occlusion (sample for color bleed).

I’m exaggerating here, but in particular the temporal factor and blurriness in UE4 makes for laggy hallucinogenic SSAO that is reminiscent of udk3. I can’t speak here and now for how precise or fast it is compared to the other methods (but at a glance it doesn’t look to be the best in either case (albeit pretty good)).

There were a couple of posts further up, from peeps who use the material editor but don’t write shaders who kinda said we should just stick with the material editor. But my mistake, I should have written “anyone” instead of “everyone”, and certainly didn’t mean to say here was saying that.

No. Just no.
I have firm belief that node-based shader editor should be a supplement, that facilitates shader creation, not complete replacement of the process.

It is not THAT hard to tweak whatever you need in the rendering pipeline right now, but saying frankly, I would prefer having an option to at least author a shader from scratch without even needing to touch the engine code.

I don’t think that built-in UE4 SSAO is inferior in any way to analogues.

Nailed it… the material editor is the blueprint equivalent but there is no proper code equivalent (custom node is super limited and working with .usf is far from great and often requires engine source modifications).

Basically UE4 hates HLSL programmers.

I think as a start it would be nice to at least be able to load .usf files from plugin or project folder :rolleyes:…