Stencil rendering in MovieSquencer

hello All.

I was watching WilliamFaucher he did great stencil tutorial. So I did the same steps added my character and test sphere, but when I render out… and check exr. The backdrop of the MTH is renderd too. That should be in the default layer as I turned it on. That’s what’s william mentioned all other meshes will be rendered in that default layer. So now

I need to select my backdrop and turn it off so it won’t render. But it shouldnt be like that. I should have exr with character and sphere as those two meshes I added to a ney layer. Shouldn’t this be renderd on a transparant BG? Or am I missing something?

If you wish to render layers out with an EXR alpha:
In Project settings search for Alpha. Rendering->PostProcessing → Enable Alpha channel support
to Linear Color Space Only
And set Frame Buffer Pixel Format to 10RGB, 2bit alpha (More display but maybe needed?)
If the EXR isn’t multilayered then I tend to turn that checkbox off in the movie render que.

Hello Scottunreal,

Thanks for your kind help. I already had these settings in my project set, but I went wrong as I was following the tut instructions, I only have one mesh I turn on multilayerd that was causing it. If you have more meshes then you turn it on. So its working now.

1 Like

When i use stencil render , the alpha of each stencil layer are not included ( pure black ), i have turn on the Alpha Channel Support to Linear color space only, and also set the frame buffer format, but still no luck with the alpha on the stencil when i put it inside Nuke, Althou, I can separate the layer using Shuffle nodes inside Nuke just Like Will did on his video, but when i switch my viewport to Alpha inside Nuke, it was all black. Any thought or idea on what i did wrong inside unreal?

Here is my setting inside Unreal:

I can seperate this using Shuffle nodes inside Nuke when Viewport RGB:

but no luck when i try to view the Alpha:

Any help will be much appriciated…

so, did you ever solve this?
i run into the same issue :confused:

I’ve also found that stencil layers are not working as intended, some objects can be added to a layer and render with alpha, while some cannot. Even rendering the same object in a different layer, sometimes the alpha is black when it previously worked.

Okay so I’ve figured out what the issue was for me. NANITE! It seems that static meshes with nanite enabled will not show up in stencil layers at all in UE 5.0. Hope this helps some of y’all.

Using the console variable “r.Nanite” set to Zero in the Movie Render Queue seems to take care of the issue with stencil layers not rendering without you having to manually disable nanite on each mesh. Hope that helps anyone dealing with this issue.


Thanks, I was getting crazy…even though I heard nanite can cause this, I didnt think that all my meshes had nanite enabled automatically.

This console command is gold!

1 Like