Is it possible to access mesh data such as vertex in shader?

[Janghoon_Kang]

I’m working to calculate curvature at static and skeletal mesh on runtime and save that data into the mesh object.
After that I would like to access that curvature data into the shader, mostly on MaterialTemplate.ush.
However, I only can access world normal or position at the material shader or material BP now.

Therefore, is it possible to access mesh data such as vertex in shader or material BP?

[Chosker]

modifying the entire engine pipeline to pass the data all the way from the mesh to the shader and output as material nodes is probably not worth the huge effort.
the easiest way would be to bake the curvature into the mesh vertex color or into a separate texture. or making a new UV channel where you store the curvature value as the UV position (you only need one axis in the UVs)
out of the 3 solutions above I’d probably pick the extra UV channel simply because it gives more freedom and versatility. for reference, SpeedTree stores all its tree branch/wind/etc properties as extra UV channels.
using the mesh vertex color has as downside that you lose a vertex color channel for vertex paint if you need that, and baking it to a texture makes it much less versatile as you need to manually pass it to the shader etc.

[Suthriel]

So it isn´t possible to calculate the curvature here in UE for models, that do not come with this stuff already baked? If it would be possible, i guess, it would also help some edge shaders for those toon shaders, as in, not only working in screen space, but for the complete mesh and make it work in raytraced reflections too.

[MostHost_LA]

If you are just interested in curvature - which is static on static models - why not bake it as a texture like we all do when applying paint/wear etc?

For morphing meshes its a bit harder, since it can’t be pre-bake.
You would really have to rewrite half the engine to do it. Parricularly because you can’t access individual vert data from a shader. And more to the case, there isn’t any built in way to tell how close one vert is to another at all form the shader side - or any engine side for that matter.

instead of loosing your mind like that. Cheat it.

bake the extreme pose curvature textures and lerp between them based on bone angle - kinda like they did for the digital human facial expression normal maps.

In fact, exactly the same concept, you just use 2 curvature maps to derive the somewhat correct approximation.
No idea how that’ll look for you, but if the intention is to increase wear/tear on the shader when a metal bar bends, this should work perfect…

[Janghoon_Kang]

@MostHost_LA

If you are just interested in curvature - which is static on static models - why not bake it as a texture like we all do when applying paint/wear etc?

For morphing meshes its a bit harder, since it can’t be pre-bake.
You would really have to rewrite half the engine to do it. Parricularly because you can’t access individual vert data from a shader. And more to the case, there isn’t any built in way to tell how close one vert is to another at all form the shader side - or any engine side for that matter.

instead of loosing your mind like that. Cheat it.

bake the extreme pose curvature textures and lerp between them based on bone angle - kinda like they did for the digital human facial expression normal maps.

In fact, exactly the same concept, you just use 2 curvature maps to derive the somewhat correct approximation.
No idea how that’ll look for you, but if the intention is to increase wear/tear on the shader when a metal bar bends, this should work perfect…

@Suthriel

So it isn´t possible to calculate the curvature here in UE for models, that do not come with this stuff already baked? If it would be possible, i guess, it would also help some edge shaders for those toon shaders, as in, not only working in screen space, but for the complete mesh and make it work in raytraced reflections too.

​​​​​

@Chosker

modifying the entire engine pipeline to pass the data all the way from the mesh to the shader and output as material nodes is probably not worth the huge effort.
the easiest way would be to bake the curvature into the mesh vertex color or into a separate texture. or making a new UV channel where you store the curvature value as the UV position (you only need one axis in the UVs)
out of the 3 solutions above I’d probably pick the extra UV channel simply because it gives more freedom and versatility. for reference, SpeedTree stores all its tree branch/wind/etc properties as extra UV channels.
using the mesh vertex color has as downside that you lose a vertex color channel for vertex paint if you need that, and baking it to a texture makes it much less versatile as you need to manually pass it to the shader etc.

Thanks for replying here. Right now, I’m using Nvidia FaceWorks to calculate the curvature first and then try to do pre-integrated skin in material part. That’s why I’m trying that way. And also, I tried to calculate curvature from custom material function and it works fine, but not sure about the satisfactory quality.

[Suthriel]

@MostHost_LA

If you are just interested in curvature - which is static on static models - why not bake it as a texture like we all do when applying paint/wear etc?

Well, if i think about the built in poly modelling in UE 4.25 upwards, then how would you give those models curvature, that you built directly here in UE? Would you need to export them to an external program for unwarpping and then baking, then re-import?

I know, that their modelling tools are in an very early stage and nowhere as good, as others, but they start becoming a thing and will most likely improve over time… but their models need materials too, and building such stuff in -engine and then being forced to go to external tools for UVs and baking the necessary masks seems a little bit counterproductive

[MostHost_LA]

@Suthriel

Well, if i think about the built in poly modelling in UE 4.25 upwards, then how would you give those models curvature, that you built directly here in UE? Would you need to export them to an external program for unwarpping and then baking, then re-import?

If you want to be a game developer you need to know how to deal with game art. You can’t expect the engine to be able to allow you to model something at the same level and quality a 3d program would.

This does mean that you most definitely will need to take what you modeled out of the engine, unwrap it properly and texture it properly.

What the in engine tools are mostly used for (so far at least) is to add terrain geometry that doesn’t necessarily need any special care when unwrapping or texturing.

The instant you need to do something interesting, like curvature bakes. You are already dealing with a 3d program, not the engine…

That doesn’t mean you can’t use the same math to find the edges obviously - it’s math.
But it does cost you in performance when there is no need for it to…

[Chosker]

@Janghoon_Kang

Thanks for replying here. Right now, I’m using Nvidia FaceWorks to calculate the curvature first and then try to do pre-integrated skin in material part. That’s why I’m trying that way. And also, I tried to calculate curvature from custom material function and it works fine, but not sure about the satisfactory quality.

The only way I know to calculate curvature from the material is using screenspace derivatives (DDX/DDY) which result in a pixelated effect so the quality is not satisfactory. I’m curious if you can find some other method

@Suthriel

Well, if i think about the built in poly modelling in UE 4.25 upwards, then how would you give those models curvature, that you built directly here in UE? Would you need to export them to an external program for unwarpping and then baking, then re-import?

I know, that their modelling tools are in an very early stage and nowhere as good, as others, but they start becoming a thing and will most likely improve over time… but their models need materials too, and building such stuff in -engine and then being forced to go to external tools for UVs and baking the necessary masks seems a little bit counterproductive

some time ago I made a curvature baker tool inside UE4 with blueprint via procedural mesh component so it’s definitely possible to do it in-engine.
but like MostHost LA suggests, anyone serious about making game art should use specialized software. the in-engine tools are good for prototyping but trying so hard to cram everything to be in-engine for the sake of convenience will just lock people into inferior software