Bad raytraced shadows with voxelised Nanite Foliage in 5.7

I’m trying to use the new voxelised nanite foliage in a cinematic project. This project uses Ray traced shadows rather than virtual shadow maps.

However, it seems that raytraced shadows still use the nanite fallback mesh, rather than the nanite assemblies used to make up the tree. This causes ugly, harsh shadows on the trunk, and inaccurate, polygonal shadows on the ground:

I have tried setting r.RayTracing.Nanite.Mode 1, and while this works on regular meshes, it does not work for the new Nanite Foliage trees.

The only solution I have found is to regenerate the fallback mesh for each tree. However, this is generated from the fully assembled tree (rather than the instanced parts) and so takes forever and fills up the ray tracing memory budget quickly.

Is Nanite Foliage compatible with ray traced shadows? Or is there a way to increase the shadow quality without regenerating the fallback mesh, and without switching to virtual shadow maps?

1 Like

nope. the assemblies are too compute intensive to ray traverse and as you noticed a full detail mesh generates too much raytracing data. simply just use vsm or deal with the polygon shadows.

1 Like

Thanks for the answer. It seems a shame that I couldn’t find this in any of the documentation, but maybe I didn’t look hard enough.

well… i dunno if this needs documentation. it works, but it is “incomplete” or low quality, cause it has those serious performance implications. coders i “talked” to on here on this topic don’t see this changing anytime soon, either. it is what it is. too complex to make it perform nice with high raytracing detail.

I’d say that if it’s unlikely to change then that means that it especially needs documentation.

It’s fine for it to be unsupported, but if the only way to find that out is to ask on a forum then that’s not ideal for anyone.

I have to say that the inconsistent documentation has been one of my biggest hurdles while learning Unreal. Not having a single source of truth makes it hard to know what advice is good and what is outdated or wrong.

I understand that it’s a bit of a behemoth to document though, I don’t envy the team responsible for it.

1 Like

Not only it is confusing, they showcase the witcher demo saying it was nanite and raytracing. But I can´t really tell where the raytracing was. Water reflections were screen space, Nanite tesselation or voxel does not cast raytraced shadows. When used with megalights’ raytracing, it will drop screen space shadows…
For us using UE5 for cinematic purposes, it is a real pitty that raytracing development has been sided. But it is specially annoying downloading the new branches after seeing the exciting stuff like tessellation to always see it doesnt work with raytracing, and this info is nowhere to be found.
I suggested long ago that at least Nanite Tesselation, when used with megalight’s raytracing, could cast VSM shadows for the tesselated parts instead of screen trace, since that would at least give us the raytraced quality for the overall shape and consisten shadows for smaller details. I wish that path was explored further.