Community Tutorial: New shading models and changing the GBuffer

Implementing a Celshading model directly into UE5.1 source. This celshading use a linear color curve atlas to drive all the values.
Learn how to set your own shading model, add a render target to GBuffer, storing information in the View struct, creating a global node for material graph. Some tips and tricks
We will see restriction about mobile and raytracing.


Hello @One3y3.
Great post, haven’t dig deep into gbuffer section first but the rest of it is just great.
So I want to point out something that bothers me:
AFAIK the only communication between base pass (gbuffer) and deferred pass(lighting) is through gbuffer shading model.
The problem is shading model encoded in gbuffer (4byte), sharing 4byte with selective output mask. (Check EncodeShadingModelIDAndSelectiveOutputMask).
Now the code in tutorial using up to 16 (with SHADINGMODEL_NUM assigned to 16).
It might be works in this case, if it’s not involving raytrace, but it might be harder to extend it beyond this. (For example Toon model for hair, etc).
Changing the number of bits allocated to store GBuffer shading model is not preferrable (due to the amount of engine files touched - integration nightmare).
So I wonder if anyone has better approach to extends this beyond shading model id.

From what I saw in the gbuffer generation, Shadingmodelid is a int value packed into a uint4 with Outputmask, which then is converted into a float in the alpha channel (because textures are only 0-1 range).
So if uint8 is 0 to 255 values, I assume uint4 is 127 values available for shading model ids.

I don’t have any problem when enabling raytracing shadow / raytracing skylight and hardware Lumen in my case.

GBuffer is the only way to communicate specific material properties to the deffered light rendering pass. There is some global struct like the view structure, but it’s common to the whole scene. The only way would be to add your own subpass in between basepass and light pass. But that’s like re-coding the whole engine and raytracing implementation :slight_smile:

Hi! great post and thank you for the tutorial


Here is the comment taken from ShadingCommon.ush

// SHADINGMODELID_* occupy the 4 low bits of an 8bit channel and SKIP_* occupy the 4 high bits

so in essence, ShadingModelID uses 1 channel on GBuffer, and in this channel shared half of them with OutputMask.
So effectively only 4 bits are allowed, which is 15 only.
I haven’t found any issue so far, it might be I haven’t used OutputMask extensively yet.

I am still struggling to find the proper solution to extend this ShaderModelID beyond 15 without not much gbuffer encoding modification. Still have so many things to learn and explore :slight_smile: