Stylized rendering (yet another). Shadow info in Surface mat, OR use diff material in shadow, OR material override in render layer

Development Rendering
,
1

Working on look dev for an animated film.

I would like to give shadows on certain objects a different texture (in the long run, a sort of stipple look). So there’s a sort of base color/texture, then using a shadow mask, a Shadow texture.

image
image836×1042 204 KB

image
image836×1048 224 KB

image
image836×1031 84.4 KB

image
image836×1048 220 KB

(obviously it’s super duper ugly currently, just trying to find the workflow right now)

I’ve seen this asked a LOT and there hasn’t been an answer but maybe there’s an alt approach…

Recognizing the challenges involved in using shadow/light information in materials that aren’t Post Process (unless there are new developments there??), I’m wondering if there’s a way to have specific objects use different materials based on a shadow mask.

Maybe a simpler way of asking:

  • can a material (or even PPM) have different attributes (color, e.g.) based on a mask generated by lighting/shadows?

Assuming that isn’t possible…
An alternate would be - is there a way using Render Layer Passes to have a material override so I could render out a pass using one set of materials, but then render out a second pass (for use in compositing) that uses a different set of materials without having to somehow manually duplicate the level and change materials on all the objects.

I’d prefer to do it realtime vs compositing together later, thinking there may be a little more experimenting I could do with the effect in engine with how the texture is being wrapped around objects, and maybe with some world position offset stuff, but if it can ONLY be done after-the-fact, then so be it.

Anyone have any thoughts/ suggestions?

(the hatching is currently based on https://www.youtube.com/@HepnerKamil nice tutorial! But it will not end up using this exact look)

2

Without doing custom render passes or modifying the engine this is basically impossible. You can get close with postprocessing by comparing the lit image with the unlit base color to create a shadow mask.

The issue is that deferred renderers don’t have access to shadow data at the material stage. In fact, that’s the whole point of deferred lighting. You create buffers to inform lighting and shadowing so that it can be applied in a single pass later on.

Forward renderers can do this type of thing though. Tom Looman has an example of this on his site.

You can recreate core shading masks in the material though, by passing light vectors to the material as parameters and then calculate the dot product of the normals against the light vectors, and add them all together.

This won’t work for cast shadows though, since they still don’t exist at this stage.

1 Like
3

Was afraid of this.

I wonder if this is the Tom Looman example you mean:

Unfortunately that method was apparently based on a ‘bug’ and is no longer an option