I’m prototyping a photography mode / ID system where players can photograph the world and essentially unlock a codex of info about subject(s) that exist in the game’s database (think pokemon snap).
When the player snaps a pic, a SceneCapture2D captures a frame, and so by reading the pixel values from the render target, i would like to get the custom stencil ids of the mesh.
in project settings: custom depth-stencil pass is already set to enabled with stencil.
the capture source of the sceneCapture2D is set to: SceneColor in RGB, SceneDepth in A.
the render texture format is set to: RTF_RGBA8 (the clear color is 0,0,0,0)
photo subjects in the scene look display correct stencil ids at runtime when setting viewport to buffer visualization Custom Stencil
currently using a simple cpp script to read the pixels from the render target but cannot seem to get stencil ids, have tried the red and the alpha channels.
I’ve seen conflicting information about whether stencil data is accessible from the raw render target. Would appreciate any clarification on the most straightforward way to implement this.
afaik scene capture will capture the rendered image. that is the albedo or color output.
the stencil is in the custom depth stencil buffer. (which needs to be set on the settings iirc).
you’re not capturing the stencil buffer, only the color buffer.
you could set your scene2d to capture the stencil buffer, and or use two secene2d.
or if you want to go into the try-hard mode:
you could forward that information if you use some sort of encoding (or a form of steganography)
for example using certain channel, or tweaking the values. or using a colormask.
i don’t know much of this, but seems pretty far fetched if you ask me.
thanks for the reply. do you know what setting is appropriate for scene2d to capture the stencil buffer? are you referring to the “capture source” setting?
i was hoping that i could get the stencil to pipe through the Alpha channel when reading the pixels of a capture. but maybe this isnt possible; this is what im trying to clarify.
most of the options don’t set anything on A, except for the 1st InvOpacity on A.
one idea, which might not work, and you’ll have to try for yourself because i’m busy, is:
add a material to the post process settings on the scenerender2d component.
and in that material read the stencil buffer and write it to the opacity output. maybe invert it (one minus node).
then the scenecapture will use that material and potentially you’ll obtain what you want.
thanks, ya i think that’s basically what ive been setting up today but reading the red channel:
scene capture source: Final Color (LDR) in RGB
render target format: RTF R8
post process material on scene capture pipes the stencil value out the emissive color R channel,
read the render target pixels’ red channel: anything > 0 should be any mesh that is part of this photo ID system (ie, has custom depth turned on and assigned a stencil value).
will report back if it works!
ps found this article not too long ago which sparked this whole experiment
