Hi,
This appears to be a topic that comes up every few years, so it looks like its time for my turn.
Goal: UE 5, Extract the image from a Render Target during runtime and store the pixels in an array for further processing in C++. Specifically, I need the scene depth, but the rgb may be useful for debugging.
Info:
SceneCaptureComponent2D w/ a Capture Source of “SceneColor (HDR) in RGB, Scene Depth in A”
RenderTarget w/ Render Target Format of “RTF RGBA16f” (I need at least the 16f or the 32f), Dimensions of 1024x1024
The render target is then passed into a C++ blueprint function library
Optimally, I could use “SceneDepth in R” + “RTF R16f”, but there appeared to be errors when processing textures not in an rbga format. I am not sure if it is relevant though.
At the moment, I am not paying any attention to the rbga order of the output, just trying to get the output in the first place.
The code is crashing when it runs line 162 (The RHILockTexture2D line). When I switch the EResourceLockMode to WriteOnly, it manages to run just fine, but the PixelArray is empty as expected. So I think there is something going wrong with the RHILockTexture2D ReadOnly mode. Any ideas as to why this is happening?
Thanks,
Glascomet
Relevant Links to older posts
UE Forum: Reading Data From UTexture2D
UE Forum: Accessing Pixel Values of Texture2D
UE Forum: How Should I Sample A Runtime-Generated Texture in All My Game’s Materials
Code
FString UMyBlueprintFunctionLibrary::MySampleRenderTarget(UTextureRenderTarget2D* InTarget)
{
int32 InSizeX = InTarget->SizeX;
int32 InSizeY = InTarget->SizeY;
struct FCopyBufferData {
TPromise<void> Promise;
TArray<FFloat16Color> StructDestBuffer;
};
TArray<FFloat16Color> PixelArray;
int32 sizeOfImage = InSizeX * InSizeY;
PixelArray.SetNum(sizeOfImage);
using FCommandDataPtr = TSharedPtr<FCopyBufferData, ESPMode::ThreadSafe>;
FCommandDataPtr CommandData = MakeShared<FCopyBufferData, ESPMode::ThreadSafe>();
CommandData->StructDestBuffer.SetNum(sizeOfImage);
auto Future = CommandData->Promise.GetFuture();
//CopyTextureToArray(InTexture, PixelArray);
if (InTarget && InTarget->Resource && InTarget->Resource->TextureRHI.IsValid() && InTarget->Resource->TextureRHI->GetTexture2D()->IsValid()) {
UE_LOG(LogTemp, Warning, TEXT("Valid, Starting Read"));
const uint32 NumBytes = CalculateImageBytes(InTarget->SizeX, InTarget->SizeY, 0, InTarget->GetFormat());
ENQUEUE_RENDER_COMMAND(CopyTexture2DToArray)(
[Data = CommandData, Bytes = NumBytes, LUT = InTarget->Resource->TextureRHI, bFlush = false](FRHICommandListImmediate& RHICmdList)
{
uint32 DestStride = 0;
FFloat16Color* DestBuffer = static_cast<FFloat16Color *>(RHILockTexture2D(LUT->GetTexture2D(), 0, EResourceLockMode::RLM_ReadOnly, DestStride, false, false));
FMemory::Memcpy(Data->StructDestBuffer.GetData(), DestBuffer, Bytes);
RHIUnlockTexture2D(LUT->GetTexture2D(), 0, false, bFlush);
Data->Promise.SetValue();
});
}
else {
UE_LOG(LogTemp, Warning, TEXT("InValid, Skipped"));
}
Future.Get();
PixelArray = std::move(CommandData->StructDestBuffer);
int32 size = PixelArray.Num();
uint32 A, R, B, G;
FString pixString;
for (int i = 0; i < 10; i++) {
A = PixelArray[i].A.GetFloat();
R = PixelArray[i].R.GetFloat();
B = PixelArray[i].B.GetFloat();
G = PixelArray[i].G.GetFloat();
pixString = FString::Printf(TEXT("Sample Render Target. i=%d:Colors A=%f, B=%f, R=%f, G=%f"), i, A, B, R, G);
UE_LOG(LogTemp, Warning, TEXT("%s"), *pixString);
}
// Return the Data
UE_LOG(LogTemp, Warning, TEXT("Sampling Texture"));
//return (FString::Printf(TEXT("Sample Render Target. Colors A=%d, B=%d, R=%d, G=%3.3f"), sizeOfImage, size, 10, 10.0));
return (FString::Printf(TEXT("Sample Render Target. Colors A=%f, B=%f, R=%f, G=%f"),A,B,R,G));
}
Crash Error (Line 162 is the RHILockTexture2D line)
Assertion failed: !bRequiresResourceStateTracking [File:D:\build\++UE5\Sync\Engine\Source\Runtime\D3D12RHI\Public\D3D12Resources.h] [Line: 270]
UnrealEditor_D3D12RHI
UnrealEditor_D3D12RHI
UnrealEditor_D3D12RHI
UnrealEditor_LineTraceTest_1026!<lambda_7025c7817cb6d771a71b75e91192b8b4>::operator()() [C:\Users\name\Documents\Unreal Projects\LineTraceTest\Source\LineTraceTest\Private\MyBlueprintFunctionLibrary.cpp:162]
UnrealEditor_LineTraceTest_1026!TEnqueueUniqueRenderCommandType<`UMyBlueprintFunctionLibrary::MySampleRenderTarget'::`5'::CopyTexture2DToArrayName,<lambda_7025c7817cb6d771a71b75e91192b8b4> >::DoTask() [C:\Program Files\Epic Games\UE_5.0\Engine\Source\Runtime\RenderCore\Public\RenderingThread.h:193]
UnrealEditor_LineTraceTest_1026!TGraphTask<TEnqueueUniqueRenderCommandType<`UMyBlueprintFunctionLibrary::MySampleRenderTarget'::`5'::CopyTexture2DToArrayName,<lambda_7025c7817cb6d771a71b75e91192b8b4> > >::ExecuteTask() [C:\Program Files\Epic Games\UE_5.0\Engine\Source\Runtime\Core\Public\Async\TaskGraphInterfaces.h:975]
UnrealEditor_Core
UnrealEditor_Core
UnrealEditor_Core
UnrealEditor_RenderCore
UnrealEditor_RenderCore
UnrealEditor_Core
UnrealEditor_Core
kernel32
ntdll