My idea would be to create a cube associated with the HAT mesh, that cube would have no texture, but it would have a Specific “Hair-Mask” material assigned.
So each Hat would be exported to Unreal with 2 materials, the Hat material and the Hair-Mask material, and each hat would have this invisible cube that will define the masking zone.
So the question is the following ; Is it possible to somehow set a material to be masking another specific material ?
Mat_Hair-Mask (from the hat mesh) would mask Mat_Hair (from the Hair mesh)
You head and hair would most likely be skeletal meshes. The hat at that point can also be one.
Only other option would be vertex manipulation via world position offset and some sort of offset mask, or outright masking off the visibility. Though this may become problematic if you have many hats of many different shapes.
In this case you would need to change the hair material from opaque to masked and then via a black and white texture tell the engine which parts of the mesh are masked off and which are still visible. Masked materials are cheaper than translucent ones.
The hair would need to probably have a second uv map that would govern the masking off parts of the hair.
Alternatively you could have a second hair mesh that would be the masked version.
That would not work, because my hats have different sizes and angles, so the mask would need to be different for each type of hat.
In my case i need to find a way to make the “Hat mesh” be a mask to hide the hairs, that way each shape of hat would mask all the hairs.
I will experiment with stencils maybe there is a way, if i really cant figure a solution, i could try to make my hairs and hats skeletal meshes, but i will try to avoid it.
Unfortunately i never solved it, instead i allowed a single item on the head, so you can either have hairs or a hat.
I still think this sould be possible somehow, but i dont really know how.
Lots of games solve this by making a special hairstyle that fits under all hats, and swapping the normal hair to that when a hat is equipped. It’s a bit odd when a char goes from having long flowing hair to a random short style, but at least it avoids clipping. If you’re using a datatable or data asset to store hairstyles tho, you could link alternate hat-hairs there, and either make an alt mesh for every style or just make generic short/medium/long variants the styles share under hats and link them.