I wrote a shader of my own ten years ago that does something similar.
The idea is to include an alpha map in each material, and add the alpha map to the blend value between terrain layers.
The grass would then have the alpha map opaque where it has “blades.”
The blending function would then make it so that:
when the blend is 100% grass, only include the grass texture
when the blend is 0% grass, only include the mud texture
when the blend is 50%, lerp between grass at 100% alpha and mud at 0% alpha
This turns out to be a single lerp based on a scaled sum of the layer blend function, and the texture alpha
The main draw-back was that it’s more work to create good alpha materials that blend well together using this method.
I believe the same kind of material could be created in the Unreal terrain material system, although you’d probably only get 3 layers out of it if you want the fancy PBR material properties.
That makes sense! I’ll try to implement it using the material editor and see if I can get it to be similar.
If not, would it be a good idea to instead model a terrain in 3ds max, UV it, and tile textures onto it in Photoshop? That way I might be able to paint the transitions with more control?