I’m prototyping a photography mode / ID system where players can photograph the world and essentially unlock a codex of info about subject(s) that exist in the game’s database (think pokemon snap).
When the player snaps a pic, a SceneCapture2D captures a frame, and so by reading the pixel values from the render target, i would like to get the custom stencil ids of the mesh.
in project settings: custom depth-stencil pass is already set to enabled with stencil.
the capture source of the sceneCapture2D is set to: SceneColor in RGB, SceneDepth in A.
the render texture format is set to: RTF_RGBA8 (the clear color is 0,0,0,0)
photo subjects in the scene look display correct stencil ids at runtime when setting viewport to buffer visualization Custom Stencil
currently using a simple cpp script to read the pixels from the render target but cannot seem to get stencil ids, have tried the red and the alpha channels.
I’ve seen conflicting information about whether stencil data is accessible from the raw render target. Would appreciate any clarification on the most straightforward way to implement this.
afaik scene capture will capture the rendered image. that is the albedo or color output.
the stencil is in the custom depth stencil buffer. (which needs to be set on the settings iirc).
you’re not capturing the stencil buffer, only the color buffer.
you could set your scene2d to capture the stencil buffer, and or use two secene2d.
or if you want to go into the try-hard mode:
you could forward that information if you use some sort of encoding (or a form of steganography)
for example using certain channel, or tweaking the values. or using a colormask.
i don’t know much of this, but seems pretty far fetched if you ask me.
thanks for the reply. do you know what setting is appropriate for scene2d to capture the stencil buffer? are you referring to the “capture source” setting?
i was hoping that i could get the stencil to pipe through the Alpha channel when reading the pixels of a capture. but maybe this isnt possible; this is what im trying to clarify.
most of the options don’t set anything on A, except for the 1st InvOpacity on A.
one idea, which might not work, and you’ll have to try for yourself because i’m busy, is:
add a material to the post process settings on the scenerender2d component.
and in that material read the stencil buffer and write it to the opacity output. maybe invert it (one minus node).
then the scenecapture will use that material and potentially you’ll obtain what you want.
thanks, ya i think that’s basically what ive been setting up today but reading the red channel:
scene capture source: Final Color (LDR) in RGB
render target format: RTF R8
post process material on scene capture pipes the stencil value out the emissive color R channel,
read the render target pixels’ red channel: anything > 0 should be any mesh that is part of this photo ID system (ie, has custom depth turned on and assigned a stencil value).
will report back if it works!
ps found this article not too long ago which sparked this whole experiment
UPDATE: i don’t think this is possible after all due to limitations with the way SceneCapture2D works / is setup.
seems like there are posts from a few years ago requesting that Scene Capture have the ability to use the custom depth / stencil buffer…
but also it seems like post process materials behave differently when applied to an unbound pp volume (vs applied to a scene capture component directly).
whats up with that? i could be mismatching any number of settings and formats w respect to capture source and render target format but does anyone have a defacto answer on whether or not scene capture 2D can capture the stencil buffer?
is it impossible to encode the stencil value of a mesh into a single channel like Red or Alpha, and then extract it by reading back the pixels of the render texture?
are there better more practical / known ways to architect a photo id system?
i think what you’re missing is setting the scenecapturecomponent to capture the “final color ldr” which is the one after postprocess.
i recommend you to test parameters around to find what best suits you, if you notice something doesn’t work for you try stuff around.
i think r8f is enough. but remember that’s a normalized (floating point 0-1) so it wont be easy to test for a == b but it’s doable.
youll probably need to divide by 256 on the post process, and then multiply by 256 when reading the rt. (256 being the count (255 being the max), i don’t remember, but it could need a bigger number)
i don’t remember a RT format that is in integer instead of floating point. but r16f is not normalized (afaik) so you won’t need the divide and multiply trick. still don just do ==, do NearlyEquals.
here’s on my game. i use the custom stencil to draw the outline using a postprocess, and it’s set to 254 on hover.