I am creating a feature for my game that requires me to have an actor’s occlusion/visibility percentage from the player’s camera.
This isn’t going to be run on Tick or every frame. It’s a player-triggered action, and the results can be slow since I won’t need them until the player has cleared the level.
It needs to be scalable since I’m not sure how many actors could need to be checked in that single frame (20+ is possible).
I just need it not to hitch or eat frames.
Things I’ve Tried:
Creating a scene capture component for each actor I want to analyze, rendering them solo, and comparing the depth with the depth of the original photo (Capturing the scene that many times in a single frame is too expensive).
Converting the actors bounds into screen space and tracing toward every pixel in those 2d bounds (WAY too many traces).
Using custom depth and stencils (Stencil overlapping kills this idea).
Dumb ideas I haven’t all the way thought through due to lack of knowledge:
Implement a shader and use raycasting or something?
Somehow gather the information by creating a SceneViewExtension and doing something before or after the DepthPass/BasePass?
This problem has two parts.
Part 1: Cache the data.
Part 2: Read the data.
The first part needs to be fast so that none of the data changes before its all captured.
I have the second part figured out, but I have no idea how to do the first part fast without performance issues.
EDIT 05/26/24: Photo of what I am trying to achieve since I seem to suck at explaining things.
GetLastRenderTimeOnScreen(). It works like WasRecentlyRendered(), except it actually takes occlusion into account (kinda, it uses the actor’s bounds so it is not perfect).
5.4 Introduced a new class called “FCustomRenderPassBase” which lets you create custom passes to send during the main rendering process. More importantly, the scene capture component was updated to allow you to do this (only base color or depth) automatically by setting “bRenderInMainRenderer” to true. Essentially, it makes creating depth masks for every actor much faster and a viable option. There is no documentation on this other than the code and this Chinese article.
So now with a way to filter out unnecessary actors and a way to check their occlusion via scene capture and render targets, I should be done right? Nope.
I ran into an issue I had not thought about…
What happens when an actor is partly outside of the screen bounds?
I am no longer able to calculate a visibility number for the actor since I only know how much of the actor IS within the screen. Not knowing how much ISN’T leaves me asking “A percentage of what?”.
I’ve tried getting all of the actor’s verts and seeing if they can be converted to screen space, but there are many issues with this. If vert A and B of a triangle are in screen space but C isn’t, I do not know how to determine how much of that triangle is visible.
your post is confusing because is not clear if you want to have the actors fade gradually like having parts of body visible and others not visible (like they are passing trough a portal half way trough) or just instantly hide or reveal them (I’m assuming like enemies or npcs not the entire level)
yeah, well most likely the actor is not occluded when offscreen. I think occluded is “geometry in front of other geometry” when calculating the occlusion if the character is in the open like in your example then is not occluded.
you can try this hack. add some geometry (plane or polygons or something) attached to the camera that is just outside the camera view so it will “occlude” the character but is not visible by the camera.
You don’t need to explain occlusion to me.
I am just trying to figure out the percentage of the actor that is on-screen (Including the parts that are off-screen to get the full percentage).
Whether parts of it be occluded or offscreen.
My struggle comes from being unable to get an accurate percent of the mesh that is off-screen since I cannot do pixel comparisons with my scene captures (bc its not in them).
What you suggested would not work since I still would not know how much of the actor is off-screen.
