You could always just attach SceneCapture to your player camera, and have it capture the scene with many shading and postprocessing features disabled, or perhaps there’s even a way to have it capture just the lighting buffer. You’d be capturing this every frame into a rendertarget, but at the smaller resolution (let’s say one quarter) to not have much of a performance hit. Then, you can just look up value of the pixel at desired screen coordinates and evaluate that with some C++ code. If the pixel value its under certain threshold, you could consider that to be a shadow.
Here are a few snippets from my scattering plugin:
UTextureRenderTarget2D* RenderTarget = UKismetRenderingLibrary::CreateRenderTarget2D(GetWorld(), 256, 256, ETextureRenderTargetFormat::RTF_RGBA8);
UKismetRenderingLibrary::DrawMaterialToRenderTarget(GetWorld(), RenderTarget, ScatterSurface.DistributionMaterial);
ScatterSurface.DistributionTextureSize = 256;
FTextureRenderTarget2DResource* TextureResource = (FTextureRenderTarget2DResource*)RenderTarget->Resource;
TextureResource->ReadPixels(ScatterSurface.DistributionTexture); //TODO Simplify? || Move to own function?
Here I am creating a render target, but in your case you’d supply the one your SceneCapture2D component outputs to. Then, I am accessing the TextureResource (the actual texture data) using RenderTarget->Resource. You can then get the actual pixels from there. It’s literally just a simple array of FColors, that’s it. You can ignore DrawMaterialToRenderTarget and the line below it. That’s how I am drawing to render target, but again, in your case, SceneCapture2D would be the thing that is.
FColor ASimpleScatter::GetDistributionMaterialColor(const FVector2D& UV, const TArray<FColor>& ColorBuffer, int32 TextureSize)
int32 i = (int32)(UV.X * TextureSize);
int32 j = (int32)(UV.Y * TextureSize);
if (i < 0) i = 0;
if (i >= TextureSize) i = TextureSize - 1;
if (j < 0) j = 0;
if (j >= TextureSize) j = TextureSize - 1;
int32 Index = i + j * TextureSize;
if (Index < 0) Index = 0;
if (Index >= ColorBuffer.Num()) Index = ColorBuffer.Num();
This thingy then returns me a color of a given pixel on the texture, if I give it UV coords, the color buffer we’ve created above and TextureSize (this assumes texture is always square, so for non square monitor aspect ratio texture, you will need separate X and Y coords).
//Check if the point is within texture color range before storing it
bool bIncludedByTexture = Point.TextureColor.R > Randomize.RandRange(0, 254);
And in my case, using the code above, I am using the final pixel value (in this case just red channel) I got from that function to drive probability of instance appearing. In your case, if you managed to actually get lighting pass out of SceneCapture, or something close to it, it’d be just about checking if the value is above or below certain threshold to determine if it’s shadow or light.
By lighting pass, I mean something like Lighting Only editor viewport mode:
I am not sure how to setup the SceneCapture2D component to capture just that, but it has tons of toggle flags for specific shading and postprocessing feature, so I’d be very surprised if there wasn’t any way to get to that. But I do admit it may even require inheriting from SceneCapture2D and doing some changes on your own to get to that. Unfortunately I don’t have enough time to give it a shot.