Hello UE community,
It’s possible actualy to create a setup for Shadow catcher ?
Example : I use the HDRI lighting & 1 static light, point light type, whit shadows.
I want to transparance on the render whit the HDRI background & Shadows on background.
For example to render :
One idea ?
You would just do a lightmass lighting rebuild with a white floor in your level and that would render the shadows onto it. Then you could snapshot that image and composite it over your photo.
Hello,
This is not the solution I’m looking for, I have to keep the “real time” side. To be able to turn around my asset. It is also necessary that I can change the angle of light and therefore shadows. It has to be dynamic.
The only thing you could do right now is a custom per object shadow. by default the shadows would be crisp and they will only really work from point or directional lights. you would have to write a custom blurring shader to blur the result.
details about this method can be found at my blog here:
http://shaderbits.com/blog/custom-per-object-shadowmaps-using-blueprints
There is no standard feature but you can build what you need using this.
Hi , Thanks for the details.
I do not quite understand what you’re getting at.
Could you, if you have time, show me directly?
In Marmoset 3, the “shadow catcher” is exactly what I want to do in Unreal.
Just recreate the steps from the blog link. It is written like a tutorial with pictures of every material and blueprint needed.
The only thing you need to add is to reference the shadow map in your ground material. Show me when you have recreated everything to that point and then i will show you the final step for your case.
Hi ,
so unfortunately this thread stops at an interesting point. I also need a shadow catcher/matte material for compositing a car in front of a backplate (I use the new Image-Plate plugin for that). Without a shadowplane the car is just floating…
The new Composure plugin shows the effect I am looking for, but i don’t understand the BP and code good enough without documentation to rebuild the effect for my application at runtime (without Composure).
I was able to get a result with the Shadowplane Material from the Augmented Unreality plugin: GitHub - adynathos/AugmentedUnreality: Augmented reality for Unreal Engine 4
Unfortunately the material needs to be “Lit” to show shadows, so it gets brighter/darker than the backplate because of the directional light. So dead-end here…
I tried your “Custom Per Object Shadowmaps using Blueprints” Tutorial (great work! Thx for that!), and with some tweaks I got some kind of a result.
As far as I understand the workflow (its always hard to figure things out just from BPs), you’re making a Rendertarget from the Light point of view with orthographic projection. This Rendertarget is used in the ShadowPreview material and should masked somehow out the MeshComponent (Plane for me) that the Material is assigned to?
Tweaks:
Results:
Questions:
This thread is 1 1/2 years old, nevertheless i hope to hear from you. The tweaked tutorial is the best solution i got so far from my investigation…But maybe you or Corentin3D came to a complete other solution…
Greetings
Sebastian
Hey,
Nice job getting the basic shading to work. The technique above is viable but as you noticed will not scale all that well with lots of dynamic objects and lights.
I am not sure why just capturing the render target dynamically was not updating and you had to set material again. Seems like it could just be a bug with the blueprint setup, unless you are relying on some secondary shader pass to convert the depth map to the black/white shadowmap result. I think in my post I was always keeping it as a depth map so it would work live in any material without any updates.
I highly recommend you take a look at the new Composure sample in the Epic Launcher. There is an example of a shadow catcher setup in there using proper lights. It is using the composure plugin but I also helped another person set up a shadowcatcher a few days ago without composure.
The gist of the more ‘robust’ way to do this is to instead use a white flood mesh and set that mesh to be only visible to a Scene Capture. he scene capture can be pointed down at the ground orthographically. Then you set any shadow casters to ‘cast hidden shadow’. Then you can hide them and the shadow will render onto the white shadow catcher ground.
The trick is making the shadow catcher ground not render in the main view. Technically it should be possible via the ‘owner only see’ flag, but that wasn’t working recently. So instead I just did something a bit hacky that checked if the camera vector was straight down (ie dot(CameraVector, 0,01) and check abs value), and collapsed the shadowcatcher in the vertex shader by adding negative worldposition.
Its also possible to handle that visibility problem more gracefully by manually hiding the shadow catcher every frame, then unhiding it just before the scene capture updates and then re-hiding it.
Big thanks for the friendly, fast and detailed response!
I’ll return to the project after the weekend and will then take a deeper look at your proposals. I’ll keep in touch with you.
Have a nice weekend!
Hi ,
so your advices were a goldmine 
I use 1 Directional Light and 2 planes, both planes are at the exact same position. One is the ShadowCatcher (with white Material and default hidden) and one is for showing the Shadows (with a Translucent and Unlit Material, that got the inverse RenderTarget Texture as Opactiy Input).
Then I have the SceneCapture on top of the planes (distance depends on the size of the planes…). The SceneCapture uses Perspective-Projection Type, SceneColor Capture Source and it ShowsOnly the ShadowCatcher Plane and the objects that should cast shadows. “Capture on Movement” and “Capture every Frame” is disabled.
I use a Cube- and a Sphere-Object for testing with activated “Hidden Shadow” flag.
Every Tick I unhide the ShadowCatcher, hide the objects (Cube and Sphere), capture the scene, hide again the ShadowCatcher and finally unhide the objects.
I can even activate/deactivate the Directional Light every Tick only for the Scene Capture, by this way I get Shadows without lighting the objects (That will might be useful for my Project).
The results are really better and I have all the freedom to tweak the shadow-settings inside the Directional Light or with the material thats holding the RenderTarget Texture.
First I was afraid, that maybe there would appear some kind of flickering caused by the hiding/unhiding, but it seems to be all good…
Here the results (Shadow Contact seems to be a bit of a problem with the Cube):
So this is my first approach. I will look deeper into the details and tweaks…
What would you say, how much impact this technique has performance-wise? I already inserted this in my main project, and sadly the fps are dropping a bit. But I also have to use a 4 Million polygon model with a complex car-paint Shader, Glass, etc… I haven’t yet searched for the performance bottleneck.
Again thanks for your help and friendly greetings from Germany
So sadly, but this seems not to be the solution for my project.The FPS drop is heavy with the complex 3D Model (4M Poly. and complex hierarchy), with a “normal” 3D Model it works fine…
I don’t quite understand right now, why the process to hide/unhide the complex model every tick is costing me 15-20 Fps alone (Without Scene Capturing!).
So AActor::SetActorHiddenInGame() uses MarkComponentsRenderStateDirty() which iterates over all Components of the Actor to set the RenderStateDirty Flag. And with a complex hierarchy this iteration costs the fps? …(The model has got around 1000 Static Meshes and also nearly the same amount of SceneComponents…)
That is strange, the cost to hide-unhide should be very minimal, unless you are hiding hundreds or thousands of assets. I would not expect polycount to affect that cost. It would make the base pass of the scene captures pretty high though.
Why do you need to hide/unhide the hipoly object? Usually it is just the simple flat shadowcatcher itself that gets hidden/unhidden. And there are ways to avoid that cost as well by using the vertex shader as I mentioned. Ie if the view is ortho top down, you can hide the mesh in the vertex shader if the camera direction Z is -1.
Hey ,
do you mean with “hundreds of assets” the count of StaticMeshComponents inside my Actor BP? Then yes, its very high with about 1000…
With “The scene capture can be pointed down at the ground orthographically.” & “if the view is ortho top down” you mean the SceneCapture positioning like in the last picture from me? So this positioning and the Projection Type set to Perspective is right, isn’t it?
I don’t quite understand what you mean with:
“So instead I just did something a bit hacky that checked if the camera vector was straight down (ie dot(CameraVector, 0,01) and check abs value), and collapsed the shadowcatcher in the vertex shader by adding negative worldposition.”
&
“you can hide the mesh in the vertex shader if the camera direction Z is -1”
You hide the ShadowCatcher by changing its position in Worldspace every Tick?.. Is the cost to hide/unhide the ShadowCatcher higher than changing the position ( i tested it quickly and it doesn’t seems to work…)
The main problem (beside the high cost of the ScreenCapture caused by the polycount) seems to be the hide/unhide of the hipoly object, so i have 2 questions/problems here:
"The gist of the more ‘robust’ way to do this is to instead use a white flood mesh and set that mesh to be only visible to a Scene Capture. "
By adding only the ShadowCatcher plane (= white flood mesh) to the “Show only Actors” list from the SceneCapture, there are no shadows visible in the RenderTarget…
So i need to add the high poly object to this list, otherwise i got a blank white RenderTarget texture…
"Why do you need to hide/unhide the hipoly object? " Well you wrote: “Then you set any shadow casters to ‘cast hidden shadow’. Then you can hide them and the shadow will render onto the white shadow catcher ground.”
If i don’t hide the object (whats then the point to use “cast hidden shadow”…), the SceneCapture renders also the object and this makes the RenderTarget texture not useable… (But maybe with a Mask?)
Sorry, if I misunderstand some things here, but I just started 2-3 months ago with Unreal and there is so much to understand and learn!
“You hide the ShadowCatcher by changing its position in Worldspace every Tick?.. Is the cost to hide/unhide the ShadowCatcher higher than changing the position ( i tested it quickly and it doesn’t seems to work…)”
You don’t need to tick anything to use vertex shader instructions. Things like camera vector are just shader constants so doing a dot product using it is dirt cheap. It would only cost a few ALU instructions, nothing on the CPU necessary.
“do you mean with “hundreds of assets” the count of StaticMeshComponents inside my Actor BP? Then yes, its very high with about 1000…”
Yes, that is way too many things to be toggling hidden on between frames. A few objects is fine, but not 1000. When I suggested hiding/unhidding I thought you were just talking about a few objects.
There may be some other ways to do this. Maybe you can enable the project setting clip plane setting and then enable the Clip Plane on the scene capture2d and adjust it so that it clips just before rendering the shadowcatcher ground.
That Clip Plane was the last thought I had yesterday before leaving. And it works, so no more hiding/unhiding the highpoly object!
But the SceneCapture is still very expensive. Is it always rendering the highpoly object and clips afterwards?
The Base Pass of the SceneCapture has the most impact, the shadow calculation for one directional light cost only 5ms.
The materials on the car also doesn’t impact the FPS much, I tried a simple grey one for everything and gained only 1-2 fps.
So Tick events go through the CPU and of course vertex shader instructions through GPU. I have to edit the Material of the ShadowCatcherPlane to implement those instructions? But I can’t follow your instruction about the camera direction… Do you mean the SceneCapture when you say Camera? Thats a bit confusing for me, but hiding/unhiding the ShadowCatcherPlane seems not to have an impact on the FPS, so that can be as it is. Still i’m curios about your way of hiding it 
If i cant reduce the impact of the SceneCapture Base Pass, i have to dig deeper and find another way.
For now the FPS without SceneCapture/Shadow is by 30…
With = 12-13 FPS…
Here some Stats:
I am glad it worked, but yes using a scene capture at all in a scene with thousands of actors is going to be pretty expensive. most of that cost will show up as ‘initviews’ cost and actually be on the CPU. But in your case the base pass is pretty high too so the raw number of polys rendered could be an issue. When you capture for the final shadowcolor instead of the depth, it actually ends up being twice as expensive because for the view frame you are rendering the mesh plus shadows. If you switch to only depth version it will help some, and then you just have to add some nicer shadow filtering to the material itself which has a small gpu cost.
If you do ‘stat dumpframe’ and search the log, you can find CaptureCommand and see the cost of the cpu side. There could be some bottlenecking going on that makes the gpu stats appear wrong.
Sadly even if you tell the scene capture to only view 1 mesh, that cost will not change as the engine is not running an optimized path for this case, it still does all the view setup as normal then masks visibility at the end.
I will have to think for a bit to see if there is some way to do this without the scene capture right now. Usually when we do this its on a very simple scene so the overhead is not so high.
Excuse me, but i was a bit too fast to say that the clipping of the SceneCapture is working… It works fine, as long as I don’t rotate Planes and SceneCapture (roll is ok, but not pitch and yaw). I update the Clip Plane Normal to the Forward Vector of my Capture Component before the capturing. Thats working, but I don’t understand the Clip Plane Base values. My first thought was, to use the ShadowCatcherPlane location, but that’s not working…
However, independent of that problem the FPS count is still problematic by using a SceneCapture, it’s a pity, that the masking of the visibility comes at the end. So the clip plane is more to understand as a masking than a real clipping?
So maybe i should take a look back to the approach of the material from the Augmented Unreality plugin?
It aligns the UV’s of the texture, so it matches with the Image Plate. Activating this, costs only 1-2 FPS:
Material:
ScreenAlignedUVsFixed:
The problem is, that the material is a normal DefaultLit one and so it’s influenced by the intensity of the Directional Light. So with the “wrong” intensity the plane is visible…:
Also i have to calculate the normal of the material per Tick, so the reflection intensity keeps steady in case of rotations. I subtract the directional light forward vector from the ShadowPlane up vector and normalize it. It works, but not perfect at extrem rotations…
It would be nice, if there would be an Unlit Shading Model (or translucent blend mode), which still receives shadows (I know, it’s a contradiction)…
So i will take a deeper look into adding a custom shading model (this is the only tutorial i found about this yet).
Sorry, if i write too detailed about all this, but maybe it helps other users as kind of a tutorial…
Here is the line about the CaptureCommand from the Log:
[2017.10.20-11.51.19:896][703]LogStats: 68.182ms ( 2) - CaptureCommand - STATGROUP_RenderThreadCommands - STATCAT_Advanced
A little update: The calculation of the Clip Plane Base/Normal values is now correct and the clipping is working.
Hmm, I have a stupid way to cal shadow catcher result(shadow only). hope this helps.
change the all shader in to grey,no texture.
render it out 2 times , frames with shadow and frames without shadow.
then divide these two images, and you get a shadow only that you can multiply to other images such as the background. if you will get some zero pixel and you might get error pixel after divide. please clamp it.
