How do I get a TArray of grayscale pixels from a Render Target

Development Rendering
, , , ,
1

I have a USceneCaptureComponent2d attached to an actor that I am programatically controlling which is set to NOT capture every frame and NOT capture on movement. I have a non-scene component that holds a pointer to this USceneCaptureComponent. Periodically I would like my component to call CaptureScene(), and then read the resultant pixel data from the TextureRenderTarget2D. Ideally, the pixel data would be formatted as a TArray(uint8) of grayscale values. Currently, I have the capture component set to full desaturation and am using a threaded FRHI ReadSurfaceData request to get the data out as a TArray(FColor) and then converting manually to to a byte array of grayscale values by doing GrayArray.Add(pixel.R) for every pixel in the FColor array. This is obviously very slow and doesn’t scale well for my application. Is there a way to get a grayscale byte array directly from a render target so I can do things like memcopys on the buffer to copy the data around efficiently?

2

I have the same issue…
Could someone please help us?

I have to get a byte array from a render target in C++, so that I can send the “picture” to other device using a sort of TCP protocol

3

unresolved for six years and still accurate: the built in ReadRenderTargetPixel family is a game-thread stall, it flushes the rhi command queue and waits for the readback before returning, so calling it per capture hitches. the non-blocking route is to copy the texture into a staging resource on the render thread and read it back through a fence, which is engine c++ territory, blueprints cant reach those APIs.

greyscale is the easy part once the samples are back: convert from the RGBA or float values yourself, Y = .299R + .587G + .114B, then push the bytes down your tcp path.

Async Render Target Reader, a render target readback plugin for Unreal Engine 5, does the staging copy for you: Read Render Target Pixels Async hands the pixels plus image size back in an On Success event and the game thread never waits for the gpu: Async Render Target Reader — Non-Blocking GPU Pixel Readback for Blueprints | Fab