Sadly Tessellation is no longer supported in UE5
See my response here Ask Unreal Anything: Rendering | June 15, 2022 at 1PM EDT - #69 by BrianKaris
What can we expect from such an alternative (if it comes)?
What are the approaches that are being looked into? (compute shader or still tessellation shader)
Wish I could say more but its too early. All I can say is the research is in the realm of Nanite, would require Nanite to be supported by the platform, would be a novel and proprietary approach, and is focused on displacement mapping as the top priority with other use cases perhaps also fitting. Sorry for being a tease but the potential solutions I’ve explored have run the gambit and I couldn’t possibly talk about details of a specific approach when I’m not 100% sure that’s where we’ll land. I’m definitely narrowing in at this point though. I’ve probably said too much already 