Reading data from UTexture2D

Development Programming & Scripting C++
1

Hi,

I am trying to read the pixel data from a populated UTexture2D in an Unreal Engine C++ project. Before i post the question here, I tried to use the method described in this link: https://answers.unrealengine.com/questions/25594/accessing-pixel-values-of-texture2d.html. However, it doesn’t work for me. All pixel values I got from the texture are some garbage data.

Here is how i do it:


I created a utexture2d (640 * 480) and initialized with blue color:
	for (int i = 0; i < 640; i++) {
		for (int j = 0; j < 480; j++) {
			int idx = j * 640 + i;
			PixelDepthData[idx].B = 255;
			PixelDepthData[idx].G = 0;
			PixelDepthData[idx].R = 0;
			PixelDepthData[idx].A = 255;
		}
	}


	// TODO Test Area
	//RequireCameraDepthMap();

	UpdateTextureRegions(
		VideoTextureColor,
		(int32)0,
		(uint32)1,
		VideoUpdateTextureRegionColor,
		(uint32)(4 * REAL_SENSE_RGB_WIDTH),
		(uint32)4,
		//(uint8*)(RealSense.GetColorDataTmp()),
		(uint8*)PixelDepthData.GetData(),
		//reinterpret_cast<uint8*>(m_pColorRingBuffer[m_iGameThreadReadIdx].GetData()),
		false,
		ColorRegionData
	);

Then, in my frame update function, I try to read the data back from this texture and then update it with it’s old values, just want to see if I got the correct pixel values from this texture using the following codes:


	UpdateTextureRegions(
		VideoTextureColor,
		(int32)0,
		(uint32)1,
		VideoUpdateTextureRegionColor,
		(uint32)(4 * REAL_SENSE_RGB_WIDTH),
		(uint32)4,
		//(uint8*)(RealSense.GetColorDataTmp()),
		(uint8*)PixelDepthData.GetData(),
		//reinterpret_cast<uint8*>(m_pColorRingBuffer[m_iGameThreadReadIdx].GetData()),
		false,
		ColorRegionData
	);

	ENQUEUE_UNIQUE_RENDER_COMMAND_ONEPARAMETER(
		FRealSenseDelegator,
		ARealSenseDelegator*, RealSenseDelegator, this,
		{
			FColor* tmpImageDataPtr = static_cast<FColor*>((RealSenseDelegator->VideoTextureColor)->PlatformData->Mips[0].BulkData.Lock(LOCK_READ_ONLY));
			for (uint32 j = 0; j < 480; j++) {
				for (uint32 i = 0; i < 640; i++) {
					uint32 idx = j * 640 + i;
					RealSenseDelegator->PixelDepthData[idx] = tmpImageDataPtr[idx];
					RealSenseDelegator->PixelDepthData[idx].A = 255;
				}
			}
			(RealSenseDelegator->VideoTextureColor)->PlatformData->Mips[0].BulkData.Unlock();
		}
	);

So, the texture was blue, but after I update it with its old pixel values, the texture is in a white color. So, does anybody know how to get the data from the utexture2d? or am I missed something here?

Thanks,
Z

2

I just want to get the depth values from the SceneCapture2D and a post-processing material that contains SceneTexture: Depth node. I need the depth values available in C++ so that I can do further processing with OpenCV. In Directx11, staging texture can be used for CPU read, but in the unreal engine, I don’t know how to create a ‘staging texture’ like Dx11 has. I can’t get the correct pixel values from the current method which makes me think I may try to access a no-CPU-readable texture.

3

Figured this out, you have to use RHILockTexture2D in the Render Thread. This platformData doesn’t give me what I want, still don’t understand what’s the purpose of this PlatformData member variable of UTexture2D. The document doesn’t provide enough information for this.

4

Hi, HolyKaisar, would you mind to post your solution, i am having the same problem reading pixels from a utexture2d, and using the example from the link of your first post only crashes the editor.
How did you use the RHILockTexture2D ? :slight_smile:

5

Hi!, I post an anwser before for another guy, but I can’t find it :p, so, here we go again.

populate UTexture2D*:



TArray<FColor> rawData;
for(int32 i = 0; i < (640 * 480) ; i ++)
{
     rawData.Add(FColor(255,0,0,255));
}

UTexture2D* texture = UTexture2D::CreateTransient(640,480);
FTexture2DMipMap& Mip = texture ->PlatformData->Mips[0];
void* Data = Mip.BulkData.Lock( LOCK_READ_WRITE );
FMemory::Memcpy( Data, rawData.GetData(), (640 * 480 * 4));
Mip.BulkData.Unlock( );
texture->UpdateResource();

now we have our texture full with blue.
Reading texture:



uint8* raw = NULL;
FTexture2DMipMap& Mip = texture ->PlatformData->Mips[0];
void* Data = Mip.BulkData.Lock( LOCK_READ_WRITE );
raw = (uint8*)Data;
// read here in low level:
//let's say I want pixel x = 300, y = 23
//basic formula, data[channels * (width * y + x)];
FColor pixel = FColor(0,0,0,255);
pixel.B = raw[4 * (640 * y + x) + 0];
pixel.G = raw[4 * (640 * y + x) + 1];
pixel.R = raw[4 * (640 * y + x) + 2];


Mip.BulkData.Unlock( );
texture->UpdateResource();

just a little remainder: you are reading data from your own texture and is consider as raw data, in other words, BGRA full 24 bits per pixel, but if you read data from an import texture, be aware you are reading diferent compressions, maybe DTX, in that case you will need read all the data and then decompress.

Cheers

1 Like
6

From utexture2d to TArray<FColor>

Thanks ZkarmaKun, your response put me in the right direction, after the code you posted i finally understood what was wrong, and why the editor was crashing.
i had some unprotected pointers…

Here is the code for others who might need it:

In my .h code i declared the following:




UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = FogOfWar)
	UTexture2D* TextureInFile = nullptr; //a pointer to be exposed to the editor, so i can select a file texture, also look at this for the compression settings https://wiki.unrealengine.com/Procedural_Materials

UPROPERTY()
	TArray<FColor> TextureInFileData; //an array of FColors

And in the .cpp inside a function:





int TextureInFileSize = TextureInFile->GetSizeX(); //the width of the texture
TextureInFileData.Init(FColor(0, 0, 0, 255), TextureInFileSize * TextureInFileSize);//making sure it has something, and sizing the array n*n
//init TArray
//What i want to do is take all the values from Texture File ->to-> TArray of FColors
if (!TextureInFile) {
	//Many times i forgot to load the texture in the editor so every time i hit play the editor crashed
	UE_LOG(LogTemp, Error, TEXT("Missing texture in LevelInfo, please load the mask!"));
	return; //<---if textureInFile is missing stop execution of the code
}
if (TextureInFile != nullptr) {
	UE_LOG(LogTemp, Warning, TEXT("Texture in file is :  %d  pixels wide"), TextureInFileSize);
	
	FTexture2DMipMap& Mip = TextureInFile->PlatformData->Mips[0];//A reference 
	void* Data = Mip.BulkData.Lock(LOCK_READ_WRITE);
	uint8* raw = NULL;
	raw = (uint8*)Data;
	
	FColor pixel = FColor(0, 0, 0, 255);//used for spliting the data stored in raw form

	//the usual nested loop for reading the pixels
	for (int y = 0; y < TextureInFileSize; y++) {

		for (int x = 0; x < TextureInFileSize; x++) {
			//data in the raw var is serialized i think ;)
			//so a pixel is four consecutive numbers e.g 0,0,0,255
			//and the following code split the values in single components and store them in a FColor
			pixel.B = raw[4 * (TextureInFileSize * y + x) + 0];
			pixel.G = raw[4 * (TextureInFileSize * y + x) + 1];
			pixel.R = raw[4 * (TextureInFileSize * y + x) + 2];
			//And then this code iterates over the TArray of FColors and stores them
			TextureInFileData[x + y * TextureInFileSize] = FColor((uint8)pixel.R, (uint8)pixel.G, (uint8)pixel.B, 255);
		}
	}
	Mip.BulkData.Unlock();
	TextureInFile->UpdateResource();
}

So why did i wanted to map the data from the utexture2d to a TArray of FColors?
Well i am working with dynamic textures and most of the pixel operations are performed in TArrays
So far i have been using this tutorial (Tutorial-Fog-Of-War) in case someone finds this code out of context

7

From my own research, if anyone else will run into the same questions: PlatformData is (probably) the data the texture was initialized with from the CPU. Since RenderTargets and and other things are only filled by the GPU, PlaformData will not help in these cases.

Since the OP did not share an solution involving the RHILockTexture2D call, here’s mine:



void CopyTextureToArray(UTexture2D *Texture, TArray<FColor> &Array) {
  struct FCopyBufferData {
    UTexture2D *Texture;
    TPromise<void> Promise;
    TArray<FColor> DestBuffer;
  };
  using FCommandDataPtr = TSharedPtr<FCopyBufferData, ESPMode::ThreadSafe>;
  FCommandDataPtr CommandData = MakeShared<FCopyBufferData, ESPMode::ThreadSafe>();
  CommandData->Texture = Texture;
  CommandData->DestBuffer.SetNum(Texture->GetSizeX() * Texture->GetSizeY());

  auto Future = CommandData->Promise.GetFuture();

  ENQUEUE_UNIQUE_RENDER_COMMAND_ONEPARAMETER(
      CopyTextureToArray, FCommandDataPtr, CommandData, CommandData, {
        auto Texture2DRHI = CommandData->Texture->Resource->TextureRHI->GetTexture2D();
        uint32 DestPitch{0};
        uint8 *MappedTextureMemory = (uint8 *)RHILockTexture2D(
            Texture2DRHI, 0, EResourceLockMode::RLM_ReadOnly, DestPitch, false);

        uint32 SizeX = CommandData->Texture->GetSizeX();
        uint32 SizeY = CommandData->Texture->GetSizeY();

        FMemory::Memcpy(CommandData->DestBuffer.GetData(), MappedTextureMemory,
                        SizeX * SizeY * sizeof(FColor));

        RHIUnlockTexture2D(Texture2DRHI, 0, false);
        // signal completion of the operation
        CommandData->Promise.SetValue();
      });

  // wait until render thread operation completes
  Future.Get();

  Array = std::move(CommandData->DestBuffer);
}

**Note: Since I didn’t care about the BGRA/RGBA mismatch, I used Memcpy directly here. If you need to process this in another order, you will need a copy loop like to the ones in the previous posts.

Be aware that, since this involves waiting for a GPU operation, this might have performance implications if done too often. There is of course also the option to wrap the downloaded array into the Future so you can do other work asynchronously.

2 Likes
8

Migrated that code to UE5.

void CopyTextureToArray(UTexture2D *Texture, TArray<FColor> &Array) {
	struct FCopyBufferData {
		UTexture2D *Texture;
		TPromise<void> Promise;
		TArray<FColor> DestBuffer;
	};
	using FCommandDataPtr = TSharedPtr<FCopyBufferData, ESPMode::ThreadSafe>;
	FCommandDataPtr CommandData = MakeShared<FCopyBufferData, ESPMode::ThreadSafe>();
	CommandData->Texture = Texture;
	CommandData->DestBuffer.SetNum(Texture->GetSizeX() * Texture->GetSizeY());

	auto Future = CommandData->Promise.GetFuture();

	
	
	FRHICommandListImmediate& RHICmdList = FRHICommandListExecutor::GetImmediateCommandList();

	{
		struct FCopyTextureCommand final : public FRHICommand<FCopyTextureCommand>
		{
			FCommandDataPtr CommandData;

			FCopyTextureCommand(FCommandDataPtr InCommandData)
				: CommandData(InCommandData)
			{ }

			void Execute(FRHICommandListBase& CmdList)
			{
				auto Texture2DRHI = CommandData->Texture->Resource->TextureRHI->GetTexture2D();
				uint32 DestPitch{0};
				uint8* MappedTextureMemory = static_cast<uint8*>(RHILockTexture2D(Texture2DRHI, 0, EResourceLockMode::RLM_ReadOnly, DestPitch, false));
            
				uint32 SizeX = CommandData->Texture->GetSizeX();
				uint32 SizeY = CommandData->Texture->GetSizeY();

				FMemory::Memcpy(CommandData->DestBuffer.GetData(), MappedTextureMemory, SizeX * SizeY * sizeof(FColor));

				RHIUnlockTexture2D(Texture2DRHI, 0, false);
				// signal completion of the operation
				CommandData->Promise.SetValue();
			}
		};
    
		new (RHICmdList.AllocCommand<FCopyTextureCommand>()) FCopyTextureCommand(CommandData);
	}

	// wait until render thread operation completes
	Future.Get();

	Array = std::move(CommandData->DestBuffer);
}
9

Regarding UE5 versions of @TheHugeManatee’s solve, this should be a reasonable in-place-replacement for ENQUEUE_UNIQUE_RENDER_COMMAND_ONEPARAMETER (without fundamentally changing the spirit of the thing):

ENQUEUE_RENDER_COMMAND(	CopyTextureToArray )(
		[CommandData](FRHICommandListImmediate& RHICmdList)
		{
		  auto Texture2DRHI = CommandData->Texture->GetResource()->GetTexture2DRHI();
		  uint32 DestPitch{0};
		  uint8 *MappedTextureMemory = (uint8 *)RHILockTexture2D(Texture2DRHI, 0, EResourceLockMode::RLM_ReadOnly, DestPitch, false);

		  uint32 SizeX = CommandData->Texture->GetSizeX();
		  uint32 SizeY = CommandData->Texture->GetSizeY();

		  FMemory::Memcpy(CommandData->DestBuffer.GetData(), MappedTextureMemory, SizeX * SizeY * sizeof(FColor));

		  RHIUnlockTexture2D(Texture2DRHI, 0, false);
		  // signal completion of the operation
		  CommandData->Promise.SetValue();
		});

…No warranties…
And yes, to the manatee’s point, that “get” will pause whichever thread you’re on so… if you’re going to do this often… consider running all of this in a thread/worker and bopping out completion events for whatever needs the pixel data.

1 Like
10

I was trying a different solution found in a different post (can’t find it now) for hours, but that one didn’t work on my packaged game. This one does.
Thank you!