Hey, I’ve made a custom snow system that my entire game depends on… I haven’t upgraded to UE 5 yet because of various hurdles. One of them is the shadows.
The old type of shadows work well with the tracks, but the landscape looks jaggy, possibly because of my material, (but it works in 4.27)
The new types of shadow works getter with the landscape but are terrible in the displaced tracks.
Another BIG question is why the new nanite tessellation is pushed out in normal space, rather than world space (like before). Do we have no options here?
I’m sorry if this isn’t specific and if I don’t provide enough information. But it’s a start, I’m just fishing for any news on UE5 tessellation, any know-how, and I’m happy to provide more info if you need!
Hey, no I’m not using heightfield meshes (you mean the one for landscape). I do use RVT over the entire map to save tracks over the entire map. When I tried them both, VHM wasn’t too happy about it.
Well, being they are supposedly just rendered as images, remove nothing.
If they were actually tessellated bits it would obviously eat them up with its mesh reduction.
Yes, I did mean the heightfiled for landscape/RVT etc.
Not sure why it wouldnt work since people are not even rendering the landacape and just using the heightfield to output the landacape into it via the RVT system.
Anyway, if i am correct and the trails boil down to a texture with a normal map - then you can play a bit with the normal map to lessen the harshness of the shadow.
If instead it is driven by actual vertex displacement, then the issue with shadowing is likely to be the vertex themselves - as in too many/too few.
Re cascade looking jaggy - that too is a “too few vertex” issue.
The shadows arent interpolated or broken up by any other filter, so each tris gets its own shading leading to potentially bad situations (often on top of mountain ridges).
When using landscape you can attempt (make copy of level first) to use the retopologize tool to lessen the effect a bit.
Btw, even with trails as displaced geometry you can “double down” on the rendering of them by creating and outputting a special normal into the landacape material from the RVT that defines the trails.
You can use that to accentuate or decrease the harshness of the shadows by essentially smoothing the normals out or making them more prominent.
This is assuming the RT you use is built with distance fields in mind…
The real challange would be to properly change the normals of the displacement so that the trail is less harsh - to do that, your displacement calls would need to first isolate the edge area they need to effect;
A good way is probably to isolate the values that are between 0 and .1 in the final displacement.
On these values, you can likely manipulate the normal.
You’d have to output the custom Tangent value out in the material - there is a cost to it ofc. So make sure to bench as always.
Some other possible alternatives are adding bump mapping via ddx/ddy
Thanks for the answers. The jagginess, I think the old 4.27 tess is subdividing so much that the vertex topography actually becomes smooth, the nanite tess doesn’t. But somehow they are fake-fixing that with the new shadow option.
I think why the heightfield didn’t work well for me is that you need one RVT for the height itself - the landscape writes to that. But then I needed to read that, slap the trail difference on top (itself coming from an RVT), write to a second heightfield with the landscape deformed - that one drowe the VHM, and that was probably a gross solution. right?
I do have access to trail normals from the system, I cranked them up and now they showed…
It better displays the artefacts with the new shadow system. It’s easier to see in a video:
“Shadow cache invalidation behavior” setting on the landscape affects this. If I select “always” the glitches go away, but yeah, can’t ship with that setting…
This is because virtual Shadow Maps are cached, and when geometry is updated, they need to be invalidated and recalculated. Changes to nanite displacement are not factored into cache invalidation. You would need to force the cache to invalidate. Unfortunately, this is usually done on a per shadow casting mesh basis, meaning you would be invalidated shadows for the entire landscape if you were to alter this setting - very expensive.
Instead, you would need to find a way to locally invalidate the cache around the places that the mesh is being updated.
One way to do this could be to increase the bounds of the skeletal mesh that is causing the displacement. The larger its bounds are, the larger an area around it will be invalidated each frame, but there is a performance cost for invalidating more of the cache each frame, so you should find the smallest bounds that doesn’t cause issues.
Edit: If the cache is reverting back after the dynamic object moves away, do you have separate static pages enabled? If so, it could be that the dynamic page is redrawing as expected, but the landscape is drawing into the static page and once the dynamic object moves away it reverts back to the old static cache. The following command controls this:
Dang that was it! Thank you! “r.Shadow.Virtual.Cache.StaticSeparate 0” and it behaves like I wanted. I didn’t change any of the settings myself, and that command was totally new to me.
Yeah I think they made that setting the default a few versions ago.
As for the last thing, nanite displacement can indeed only offset in the normal direction. I doubt that will change any time soon.
I mean, its displacement… if you access it from the shader directly you can do whatever. Always.
Try breaking into the engine’s associated USF.
Theres bound to be one that specifically refactors nanite/landscape
If you alter it, then you need to remeber to do so every time you update the engine or validate engine files.
Normally, you pull the git, modify the git, build your own engine, and all of thaf instead since projects should all be locked into a specific version by the time you want to ship.
Btw, viable alternative to this is not to use the landacape.
You can do the exact same thing on meshes. And unless nanite is locked for meshes too then the displacement should apply however needed.
Additionally.
You shouldnt rely on tessellation anyway because It’s mostly severely outdated BS in 2024…
If you go the mesh route and make proper meshes you will never have any issue regardless of what epic does - or have issues on any engine you try for that matter…
Okay, I gave VHM another try… I think I figured it out this time! A year ago I couldn’t. It’s possible there’s more snags ahead but this is very promising!
@MostHost_LA haha, you are being diplomatic with me, I know tessellation has been outdated BS for longer than that
I’m a concept artist who works with Blueprints only… I was always hoping my prototype could be redone by a coder-wizzard properly at some point. And for a long time I had a capable friend who were about to do that, like you describe, but there’s no telling when/if that will happen now.
If I can go over to use VHM that’s great. I could finally move to UE5.
Personally, I think it works best.
Though usuage may vary between engine version(s).
In the end you are basically always at the mercy of the engine unless you make completely custom stuff.
So… the real reason behind using the heightfileds is that it is the last thing they worked on which works.
Lets be real, it’s not like they ever supported their own stuff at all since they cant even fix metals rendring as black by default in however many years that ue4 has been out… still.
It works, it tested faster, and it provides hooks into other concurrently avaliable engine bits…
Does it work better than just making custom reduced meshes?
That’s questionable on a per project basis.
From what I have seen, yes.
And the bonus is you can move your code and stuff to just about any engine that supports similar Render Target caputre systems.
Ps.
I did notice your trails disappear over time, so I’ll mention this for you.
Output a niagara mesh as the foot print.
Make that mesh only visible to the render target.
Remove the fadeout/removal times off that mesh.
Any time you move around the original print is recaptured, making your trails persistent until Niagara decides it has too much stuff and culls the older ones out.
I got basically everything working with VHM… only to discover VHM don’t support shadows!? Receive yes, but cast no. So VHM are basically useless if you make hills, or anything protruding that should cast a shadow, which I thought they were for.
I do use a whole grid of rendertargets for my trails. And I don’t actually use a scene capture, I just stamp a material, I imagine that is faster.
I’ve heard niagara is the way forward… but I’m not up for a complete redo without a real programmer to back me.
No sadly not. I mean the faces facing away from the light are shaded, but that’s the full extent of shading
Am I mistaken or should I make a ticket or something?
Nope, nothing works. I explored nanite also quite a bit. No viable options. Before admitting complete defeat though, I thought of trying VHMs in 4.27 again… and they do work, as does everything else! Migrating to 5 wasn’t a option, but switching to VHMs in 4.27 is not too bad either! Something good came of it.
4.27:
