Resource->ReadPixels is very slow... faster way?

[anonymous_user_69b87708]

I have set up a 2DSceneCapture component and using c++, I access the render target and read the pixels, like so:

void MyActor::UpdateBuffer( )
{

TArray<FColor> ColorBuffer;

if (RenderTarget != NULL)
{

	FTextureRenderTarget2DResource* textureResource = (FTextureRenderTarget2DResource*)RenderTarget->Resource;

	unsigned int ReadPixelsTimer = clock();

	if (textureResource->ReadPixels(ColorBuffer))
	{
	}

	long ReadPixelsDuration = clock() - ReadPixelsTimer;
	GEngine->AddOnScreenDebugMessage(-1, 5.f, FColor::Red, FString::Printf(TEXT("ReadPixels run time: %d"), ReadPixelsDuration));
}
}

I notice that my ReadPixels takes ages to complete… So, I would like to know if there is a faster way that I can do this?

What I really want, is to access the camera view data, but as I can’t find a way to do that, I resorted to the RenderTarget method as described here: A new, community-hosted Unreal Engine Wiki - Announcements - Epic Developer Community Forums

I’ve also read this post: Accessing pixel values of Texture2D - Asset Creation - Epic Developer Community Forums but I have no idea how to get MyTexture2D.

Any advice?

[flipswitchingmonkey]

Have you ever worked something out? Because I have the same problem, ReadPixels() is just crazy slow makes up for 80% of all the time my entire function uses. There must be some way to get to the raw data quickly?

I tried using ReadPixelsPtr() instead and then copy the array around, but that is just as slow.

[anonymous_user_69b87708]

Nope. No further. Unfortunately, the support for this engine is not very good so despite my best efforts, I had to give up in the end. Let me know if you figure something out!

[flipswitchingmonkey]

Well the only thing I noticed was an absolute massive difference if HDR was activated in the Render Target. That slowed things down a lot.

You could also look into running things asynchronously using Async, but that may not be viable, depending on what you are trying to do.

Otherwise, no, it seems we are stuck with ReadPixels()…

[anonymous_user_43dfeef7]

I know this question is getting quite old, but I showed up when I was looking for a solution to the same problem.

I think found a way around this by modifying ReadPixels so it does not block the game thread. I’ve added a description to the wiki (https://wiki.unrealengine.com/Render_Target_Lookup) in case anyone is interested.

[qiuwch]

I also looking for a faster solution than ReadPixels, this is really important for many tasks.

[anonymous_user_6b572aea]

I have not found a faster solution than ReadPixels, but, as I explained in my own post here, you can speed ReadPixels up a lot by disabling HDR on the render target and setting the render target’s resolution to match the active camera’s resolution. Otherwise, the renderer has to re-allocate resources every time it reads from the render target.

[Spiris]

Great article, It was actually a launch point for this class in my code. Which I also wrote an article for. Enjoy!

A new, community-hosted Unreal Engine Wiki

[Hongj990]

Hey Guys. I share my solution.

Did you guys set the “bGPUSharedFlag”?? This value is belonging to “UTextureRenderTarget2D” class.

In my case, when I set this value to “true”, ReadPixels() was faster than before!

This is my code. I sincerely hope it helps.

[RC_Reshi]

Hey, being way late to the party I just wanted to post this, where I basically brought together everything I learned following the discussion here and on other forums, for my own projects, as I think that it may help others who want to do something similar and may or may not have less experience with UE4. This was the case for me when I read through the discussion here.

In the repository I explain a way to capture images from SceneCapture2D components to disk from scratch, giving above 30 fps even for complex scenes (dependent on your hardware).
I initially framed this towards generating data for machine learning, but of course it is usable for any application.

[MrGoatsy]

What file did you change this in?

[lehuan5062 (Huân Lê-Vương)]

I have 31 2D scene capture components running. Getting the resource from the rendering thread boost my framerate from 11.3 to 11.7 and reduce RAM usage from 9.2 to 8.1.

FTextureRenderTargetResource* resource;

ENQUEUE_RENDER_COMMAND(GetResource)([this, &resource](FRHICommandListImmediate& CommandList) { resource = TextureTarget->GetRenderTargetResource(); });

static TArray<FColor> colors;

FlushRenderingCommands();

if (resource) resource->ReadPixels(colors);

[christofpflumm]

I am aware this is a very old post, but it popped up during my search. As I found a solution how to read rather efficiently from a render target using Niagara and blueprints, I’d like to share it here in the hope someone can use parts of it. As an alternative to my description, you could look at this website from Nicholas Chalkley (I could not work out how to compile his plugin and did not want to go down that route, but for someone else it might be a solution?)

What I want to do is sum all the pixel values in a render target. I have enemies that drop virus stuff and I write that to a render target. My goal is to get the “infected area”. I want to do this continuously, so it has to be efficient enough.

I’m not using c++ so want to do this in blueprint. There are a few nodes to read from render targets

but those are terribly inefficient (which their documentation clearly states). The issue seems to be that reading from the GPU has to be done asynchronously (that’s also what the above mentioned website addresses).

Now to what I’m doing to read from a render target sufficiently fast for my application. The inspiration came from one of Ghislaine girardot’s youtube tutorial, where around 3:10 it is stated that Niagara can asynchronously read from the GPU (btw I think he has some great videos about Niagara). Next ingredient is that you can send back data from Niagara to blueprints, see for example this youtube tutorial.

So I set up the following Niagara system, starting from empty emitter, setting it to a GPU emitter:

I spawn only one particle using “Spawn burst Instantaneous”. Right after exporting the particle’s data, it is killed, so I have to trigger the Niagara continuously (see further down how I do this). I created two particle attributes which I export:

The “Callback Handler” is set via blueprint (see below, it is also explained in various tutorials how to set this). To actually calculate the sum over the render target, I use a scratchpad (“Calc Landscape State Areas”), here’s the first part of it:

Basically, it read the pixel values from a render target using the “Sample Texture 2D” function. See this youtube tutorial explaining for loops in scratchpads. Please note the “Map for” clearly states it is experimental. However, I’m quite sure that someone knowledgeable in custom hlsl in Niagara could do this better (and probably more efficiently?) in an hlsl node.

I get UV coordinates from the current loop index with the “Convert Index to 2D Lookup” node and I add half a pixel offset to this in order to get rid of the interpolation (I don’t know how to force reading of the nearest neighbour, and adding the 0.5 seems to work). In each loop, the values read from the render target are added to the respective local variables (infection, scorched etc). Those variable’s default values are set to zero.

After the for loop

I divide by the number of pixels in the render target (to normalise the sums) and write into the particle attributes (“landscape state areas 1/2”) which are exported to the blueprint. The scratchpad “Texture Sample” input has to be set to the respective render target:

Now for the blueprint setup. It is a simple actor with the above described Niagara system attached to it, I also have the “Auto Activate” of the Niagara system set to false because I trigger it in the blueprint:

In begin play, I set the callback handler (which is used in export particle data in the Niagara system):

The variable “BP Callback” is a user parameter (of type object) in the Niagara system:

The following timer is used to trigger the Niagara system which calculates the areas. The “Reset system” triggers spawning of the particle in the Niagara emitter and then the calulcation/export.

You can see there’s a “Run Landscape State Areas Materials” function which I describe further below. I found that reading from the render target the way I describe is only fast enough for RTs up to 128x128 (perhaps 256x256? You have to test this yourself). To solve this, I reduce the size of my render targets using materials (see below).

After the export is triggered in the Niagara system, the blueprint has to receive the data. This is done by implementing the “Event ReceiveParticleData” function from the “Niagara Particle Callback Handler” interface:

As I have one particle only, I can get my data out of the position and velocity vectors of the particle data with index zero (position and velocity are set to output the landscape state areas in the Niagara system).

Now to the “Run Landscape State Areas Materials” function. I have a 2048x2048 render target which cannot be summed in one go (it causes huge hickups). So what I do is to use a material which reads a render target, sums over 16 pixels and writes the result into a new render target. I do this multiple times to go from 2048x2048 to 512x512, then to 128x128 etc. I set up different render targets of the respective sizes and corresponding material instances to set the input render target, then render the materials using the following macro:

The material to reduce the size is a post process material:

First, I calculate U and V coordinates for the pixels surrounding the current pixel:

Then, pixels are read from the 16 pixels and summed row-wise:

and

The output of the for rows are then summed and go into the material’s output:

The material function to read from the higher res RT (higher res RT would be for example 512x512, the render target that is written to would be 128x128):

There’s an option to apply a “water mask” which I have in the alpha channel. I want to exclude infection that is in the water, I only want to have the infected ground area. This mask is only applied when sampling from the highest res render target (going from 2048x2048 to 512x512). For the other reductions (512x512 → 128x129 etc), the mask is not used because it would introduce inaccuracies (due to the reduce water mask resolution)

I am not 100% sure why the for-loop in the Niagara system scratchpad is not efficient enough to sum over the 2048x2048 RT, I would have guessed it is complied to GPU code and should be efficient even for bigger render targets. After all, I do the summation using the (rather awkward) material setup I described which is running on the CPU and works perfectly fine… But I really don’t know how the Niagara stuff works in detail, so I probably miss something. It could very well be that a custom hlsl node in my Niagara system could replace all the complicated material setup I described.