Shadow rendering on dynamic objects

Hi,

As I was figuring out why shadow rendering takes a long time when object counts increase I found a very peculiar thing. I discovered that the shadow light frustum is projected once per object for each light.

in pseudo code:

For each shadow casting light
	Clear the 2k x 2k shadow buffer
	for each object
		Render object to the shadow buffer at assigned tiles im the shadow buffer
	for each object
		Render the shadow light frustum to the view depth stencil to discover pixels affected by the light
		Apply light calculation bsed on the stencil values
		Clear the stencil

This is a totally non-optimal way of doing this and absolutely not scalable. I would rather first clear the stencil, then render all objects to the shadow buffer and then project the shadow caster frustums on the view and mark the affected
pixels with stencil. Then I would apply the lighting based on the stencil values.

in pseudo code:

For each shadow casting light
    Clear the 2k x 2k shadow buffer
	Clear stencil
	for each object
		Render object to the shadow buffer at assigned tiles im the shadow buffer

	Render the shadow light frustum to the view depth stencil to discover pixels affected by the light
	Apply light calculation bsed on the stencil values
	Clear the stencil

Note how in the second implementation there is just one stencil clear and we do not apply lights for each object.
My qusetion is that is there an implementation for the second approach? If not I will write it. Why does the first implementation even exist?
Does it have something to do with the fact that this way we can select objects that do not receive shadow? Is there a check box for that?

Hi Sharman, can you please clarify which flavor of shadows you’re referring to (e.g. CSMs, per-object), and perhaps point to some relevant lines of code, just so we can be clear what we’re talking about here? Also it would be useful to know the light type and whether it’s movable, static or stationary.

In the case of per-object shadows we need to do a depth-fail stencil pass for each shadow caster in order to find the pixels which intersect the shadow frustum.

Also could you please tell me how to compute the world-to-light and light-to-clip (projection) matrices from FLightSceneInfo?

Hi Sharman,
The shadow projection algorithm varies different depending on whether these are stationary or movable lights. With a stationary light, per-object shadows will be rendered for dynamic objects in the scene.The stencil test described in your first example is actually an optimisation to avoid projecting the shadow to all pixels on the screen. The cost of the stencil ops will generally be much lower than the cost of projecting the shadow to those pixels.

If you want to try whole scene shadows then you can use a movable light and enable cascaded shadow maps on it. That may be cheaper in scenes where you have very large numbers of dynamic objects. Note that objects with inset shadows (e.g. characters) will still get a per-object shadow unless you disable inset shadows.

There’s some good high level documentation on the various shadow mapping methods here:
Lighting and Shadows

Hi,

I’ll try to clarify a bit. This is the function call list:

FDeferredShadingSceneRenderer::Render()
  FDeferredShadingSceneRenderer::RenderLights()
    FDeferredShadingSceneRenderer::RenderProjectedShadows()
      FSceneRenderer::RenderProjections()
	    FProjectedShadowInfo::RenderProjection()

As far as I’ve understood all shadow casting lights follow this path. In our scenes we can have a directional light as well as spotlights, all of which can cast shadows on dynamic objects.
You mentioned per-object shadows. How can I make it so that they may affect every pixel drawn in the opaque pass, the second pseudo-code case in my earlier post? We do not want per object shadows but instead we need to consider all pixels drawn in the opaque passes.

I forgot to mention. These lights are either stationary or movable as static lights can’t cast shadows

Hi Ben,

Maybe I’m misunderstanding something but as far as I can tell from the code my second pseudocode is not even supported in 4.12. Please have a look at
FSceneRenderer::RenderProjections()

// Project the shadow depth buffers onto the scene.
for (int32 ShadowIndex = 0; ShadowIndex < Shadows.Num(); ShadowIndex++)
{
	FProjectedShadowInfo* ProjectedShadowInfo = Shadows[ShadowIndex];
	if (ProjectedShadowInfo->bAllocated)
	{
		// Only project the shadow if it's large enough in this particular view (split screen, etc... may have shadows that are large in one view but irrelevantly small in others)
		if (ProjectedShadowInfo->FadeAlphas[ViewIndex] > 1.0f / 256.0f)
		{
			ProjectedShadowInfo->RenderProjection(RHICmdList, ViewIndex, &View, bForwardShading);
			if (!bForwardShading)
			{
				GRenderTargetPool.VisualizeTexture.SetCheckPoint(RHICmdList, SceneContext.GetLightAttenuation());
			}
		}
	}
}

Here Shadows is an array of objects which means that whatever the lighting setup is, whole scene or something else, the shadows will always be applied on a per-object level, right?
I am very aware what the stencil thing does there and as you can see I’ve presented a similar technology in pseudo-code number 2, bu this time instead of projection the frustum per object like presented in

it could be projected per light on the screen. In our application we do not require a per object shadow casting capability. We want opaque objects to be shadow casters. I’ve written code based on pseudo-code
two in an other engine and it’s working very well. It’s hugely much more efficient than the per object approach.

Please correct me if I’m wrong in understanding that when a light is a dynamic light, shadows are still rendered per object. I’ve put break points in VS and I can verify that with dynamic shadows the
loop presented above is executed.

Here Shadows is an array of objects
which means that whatever the lighting
setup is, whole scene or something
else, the shadows will always be
applied on a per-object level, right?

No, if you’re using whole scene shadows (e.g. CSMs) there will be one FProjectedShadowInfo per shadow cascade. There is no per-object projection in that case. The frustum used for the projection is the cascade’s frustum extruded along the light direction.

Please correct me if I’m wrong in
understanding that when a light is a
dynamic light, shadows are still
rendered per object.

If you have a movable light and CSMs enabled, the shadow projection will be done per-cascade as described above. The only exception is in the case of inset shadows, as mentioned above. These are still projected on a per-object basis.

Hi,

Another question. would this kind of an optimisation be ok: In ShadowRendering.cpp: 2529 the stencil setup looks like this:

// no depth test or writes, Test stencil for non-zero.
RHICmdList.SetDepthStencilState(TStaticDepthStencilState<
	false,CF_Always,
	true,CF_NotEqual,SO_Keep,SO_Keep,SO_Keep,
	false,CF_Always,SO_Keep,SO_Keep,SO_Keep,
	0xff,0xff
	>::GetRHI());

and later on there is a clear at around line 2659:

// Clear the stencil buffer to 0.
RHICmdList.Clear(false, FColor(0, 0, 0), false, 0, true, 0, FIntRect());

What if the stencil values were set to zero while rendering? Wouldn’t that make the clear unnecessary? We tried it out and saw huge perf. savings, but I just want to verify that we aren’t doing anything silly here. This is our stencil setup:

    // no depth test or writes, Test stencil for non-zero.
    RHICmdList.SetDepthStencilState(TStaticDepthStencilState<
    	false,CF_Always,
    	true,CF_NotEqual,SO_Zero,SO_Zero,SO_Zero,
    	false,CF_Always,SO_Zero,SO_Zero,SO_Zero,
    	0xff,0xff
    	>::GetRHI());

To clarify, I’m talking about stationary shadow casters here (per object stuff). The clear is a huge bottleneck on ps4, I’ve verified this using razor. I guess it’s the sync after the clear that takes so long but with this new approach there is no need for the clear.

Hi,

Thanks, that clarified things a lot! One more question, can the spotlights also be whole scene shadows? I wasn’t able to find settings for those

Yes, movable spot lights will use whole scene shadows except in the case of inset shadow casters.

Thanks Ben,

That’s exactly the setup we are looking for. Basically what I’ve gathered from this is that when there are plenty of shadow casting lights and a good bunch of dynamic objects, it’s cheapest to use dynamic lights.

I will test with these settings and hopefully we can get the frame rate up again. Thanks once more.

Basically what I’ve gathered from this
is that when there are plenty of
shadow casting lights and a good bunch
of dynamic objects, it’s cheapest to
use dynamic lights.
Yes, although but it depends on the number of shadow casting dynamic objects vs static, and even the size of the objects in question. As always with these things, it’s worth profiling to see.

Yes, I think your optimization a good one. The stencil test should touch all the pixels which were previously written, so just resetting the stencil during the test seems like a good idea.

Just out of interest, you mention that this was a huge bottleneck, can you give some more detail (e.g. how many milliseconds roughly)? How many of these per-object shadowmaps are you rendering per frame - presumably quite a few?

In any case, we’ll probably look to bring this into a future release. Thanks for the heads-up.

worst case scenario in one level we had 13ms consumed on this. I don’t feel comfortable sharing the trace here though. I’m seeing 0.16ms shader clears in the trace so if there are 10 objects for 1 light that’s already 1.6ms. In our scenes we can have a good bunch more than 10 objects and possibly 2 lights so this adds up quickly.

I’m still struggling a bit with the fact that the stencils need to be applied per object. I understand why there has to be 2 applys per light when the light is stationary, but that per object thing is still difficult to understand. I’ll try to do that differently and will let you know if I’ll get it working

In the stationary light case, we render per-object shadows to provide shadowing from dynamic geometry onto the rest of the scene. We also render out pre-shadows for stationary lights in order to provide shadows from static geometry onto dynamic objects.

In both cases, the stencil test is used to find the intersection of the shadow frustum with the shadow receivers, so we only need to run the projection pixel shader on those pixels which require it.

Thanks for the info on the timings by the way. I just tried locally via and got similar results with a test level on both PC (AMD/GCN) and PS4. The shadow projection cost on PC came down from 2.5 to 0.9ms for a scene with 35 shadow casters, so a nice win overall.