Tessellation for hair rendering (isoline in hull/domain shader)

I got hints about hair rendering is successfully done programming by using tessellation in DirectX11.
I used SetGlobalBoundShaderState and mapping Vertex, Geometry, and Pixel shaders to make sure they worked, and they worked well.
Unfortunately, only the 3 shaders above which worked. I need to use domain & hull shader so I am able to control tessellation and isoline primitive generation.
I tried to inherit a class from FVertexFactory/FVertexLocalFactory (https://docs.unrealengine.com/latest/INT/API/Runtime/ShaderCore/FVertexFactory/index.html) and override SupportsTessellationShaders. No compile error but I don’t get what I expected.

  1. is my approach right to do it in UE4? if it is wrong, what approach should I try in UE4?

  2. can I get information about top view of UE4 technical graphics rendering, especially in shaders (from UE4 to USF) processing?

  3. how is the good practice of debugging shaders (the USF parts)?



Bajat - Batavian Studio

is my approach right to do it in UE4? if it is wrong, what approach should I try in UE4?

SetGlobalBoundShaderState is for global shaders only - those which don’t depend on a mesh or a material.

Tessellation as implemented in UE4 is a material feature, the material requests a tessellation amount and displacements. As a result, it’s the responsibility of the material’s pass-specific shaders (FMeshMaterialShaderTypes, like TBasePassHS) to implement the tessellation parts.

What you are trying to do requires support for a different primitive type - lines. I think you will need to create a new vertex factory based off of GPUSkinVertexFactory for this. The vertex factory will need to declare it’s primitive type (triangle or line) and the tessellation shaders like FlatTessellation.usf need to pick the right topology based on that.

I’ll be blunt, this is going to be a lot of difficult work, and probably not something to tackle if you are new to UE4. A much easier implementation is to just have very high poly hair with triangle strips, with discreet LODs to lower polycount when you are further.

can I get information about top view of UE4 technical graphics rendering, especially in shaders (from UE4 to USF) processing?

It’s hard for me to answer such a general question like this, as it’s such a wide question. But I can answer specific questions much better.

Short summary - a UMaterial compiles many shaders to implement the various rendering features. Vertex factory types are used to implement different mesh representation, for example particle sprites or rigid meshes or skinned meshes. Mesh material shader types are used to implement forward passes, for example the base pass or a shadow depth pass. Vertex factories are responsible for consuming the vertex inputs from the vertex buffer, and turning it into general vertex data through interface functions like VertexFactoryGetWorldPosition. That way the mesh material shaders can access world position without knowing what vertex factory is being used, they are decoupled.

how is the good practice of debugging shaders (the USF parts)?

I generally will output the value I’m investigating to the screen, and recompile shaders as I make edits to verify the results. When editing global shaders, ctrl+shift+. will recompile all shaders dependent on the modified files quickly. However for material shaders it will take a long time, it’s much faster to just iterate on one material by clicking the Apply button in the material editor after saving the shader changes.


We think this post contains useful information which we would like to share with our public UE4 community. With your approval, we would like to make a copy of this post on the public AnswerHub which includes the discussion but strips out your username and company name. Please let us know if you are okay with this.


Hi Sean,

Thank you for your fast response.
We want to send you the content but unfortunately we can not do it.
Then, Okay, you may make it public by stripping out the confidential information.

And also I cannot open this page (https://udn.unrealengine.com/questions/117818/vertex-factories-and-vertex-shaders.html), is it the page still exist?

Thank you.

Hi Bayu,

Thanks for your responses. It appears the page no longer exists. Was it a question submitted by your company?


Yes, it is submitted by our company.

Hi, @daniel.wright !
Thanks for replying my question and I am trying your suggestions.
I try to make vertex factory by inheriting FGPUBaseSkinVertexFactory.
In my case, how I suppose to get Render Target and observe the shaders which I have implemented.


Sorry for the delay. The render target is setup by the pass that the mesh is being rendered in - the base pass. You don’t control that when implementing a new primitive type. To use the vertex factory you have created for rendering (and their corresponding shaders), you have to create an instance of the vertex factory and submit it to the renderer in the FMeshBatch which FPrimitiveSceneProxy::GetDynamicMeshElements requests.

Yes, I think we could share this to AnswerHub, and might enrich the solutions, and it is OK without my username and company name since there is no confidential information here.

Sorry for late reply. So far, I have implemented the c++ part and usf separately. But right now I am unable to attached our resources yet, I am sorry, I will keep update our progress on these.
Recently, Trello I found status changed from wishlist/backlog to May/June/July (in progress, I think) but I found no further description. What approach/method does epic use to develop it?

I plan to work on a hair shading model in the next few weeks.

The plan is to implement a Marschner/dEon model for direct lighting

Wow, it is cool!
by the way, can I follow your work? is there any in epic’s github?

Everything checked in to our perforce repository is updated to github. You could follow checkins I make I suppose. Right now I’m working on motion blur, after that I will be on hair shading. I typically don’t check things in half way finished states. Other programmers work differently but typically I keep stuff local and only check in when I have it finished and polished. That being said you probably won’t see any hair changes for a couple of months.