I’m targeting mid-end PCs and at the same time I want graphics to be as good as I can get (for a mid-end PC of course). My main issue are “3D” materials, in my case various brick tiles and pebbles (I have urban scene). Some of them look painfully flat without effects. Also the lightning is completely static and I’m planning on using decals My ideas are: use POM, use tessellation or use none of them and rely on normal maps
For POM, to me it seems like the best quality-performance solution, but I heard that this method isn’t good for objects that you can investigate closeup and on extreme angles. I’m also concerned about combining it with baked lightning and decals since it’s just a shader effect
For tessellation, I think it doesn’t have problems with baked lightning and decals and looks great. But I’m concerned about performance cost on mid-end PCs (my level has urban set and every house has some kind of 3D material like concrete panel). And also most of my meshes are flat and I’d need to intentionally make a flat surface as dense as detailed models, which seems like an overkill of me
For static meshes your ideal option is to just use more geometry in LOD0 to begin with instead of relying on shaders to add detail. This is practically always going to be the best possible option from a quality/cost perspective. But as this isn’t always convenient and so a more practical answer is to use every option you have, depending on what the task calls for.
You’re going to encounter a lot of annoying issues using either option.
Tessellation is reliant on the underlying geometry so it is often useful to manually “pre-tessellate” areas that require more detail when modeling your mesh. Otherwise you can sometimes find yourself throwing a ton of geometry into places where it is doing virtually nothing. Because of the way it divides up the geometry, it also works better on natural surfaces like stonework, cliffs, rocks, etc. Surfaces where the crazy topology isn’t going to cause noticeable artifacts.
POM on the other hand excels on surfaces with a lot of man-made detail (buttons, knobs, sockets, paneling/grating, etc.) because it is not limited by geometry. Unfortunately as you noted the effect is more expensive and and has artifacts at glancing angles. It is also a pixel shader so it gets cheaper the less pixels are on screen.
Both have problems with shadowing and neither can change the collision of the mesh so you’ll end up with objects/characters sinking into surfaces or floating in space that are basically unfixable. Both can run the gamut from cheap to obscenely expensive depending on what you are doing.