How to transfer UTextureRenderTarget2D to CPU memory in C++ ?

[Rominitch1 (Rominitch)]

Hi Devs,

I’m trying to make a thumbnail for my save system based on latest screenshot like many games do.
To do that, I created my own APawn (see below). I created both methods:

• captureScene() which use USceneCaptureComponent2D to fill my UTextureRenderTarget2D
• dumpCapture() which read UTextureRenderTarget2D (GPU data) to write a CPU buffer

I connect the both methods to a GUI button

Currently, if I look my ingame pawn inside editor, all looks good and texture target has a good picture.

But when I press my button, I read a black image which is not my clearColor of this buffer and no error return by API.

In this code, I make a full capture every time to fill this cursed UTextureRenderTarget2D (not optimized I know but is not my issue for now).

Please help me to find my bug !!

Thanks !!!

I’m on UnrealEngine 5.2.0 official package.

Code:

APaysanPawn::APaysanPawn()
{
	// Don t rotate character to camera direction
	bUseControllerRotationPitch = false;
	bUseControllerRotationYaw   = false;
	bUseControllerRotationRoll  = false;
	
	const float radius = 80.0f;
	SphereCapsule = CreateDefaultSubobject<USphereComponent>(TEXT("SphereCapsule"));
	SphereCapsule->SetSphereRadius(radius);
	SphereCapsule->SetRelativeLocation(FVector(0.0f, 0.0f, radius));
	SetRootComponent(SphereCapsule);

	NavMovement = CreateDefaultSubobject<UFloatingPawnMovement>(TEXT("NavMovement"));
    NavMovement->bConstrainToPlane         = true;
	NavMovement->bSnapToPlaneAtStart       = true;
	NavMovement->SetUpdatedComponent(SphereCapsule);

	// Create a camera boom...
	CameraBoom = CreateDefaultSubobject<USpringArmComponent>(TEXT("CameraBoom"));
	CameraBoom->SetupAttachment(SphereCapsule);
	CameraBoom->SetUsingAbsoluteRotation(true); // Dont want arm to rotate when character does
	CameraBoom->TargetArmLength = 3000.f;
	CameraBoom->SetRelativeRotation(FRotator(-60.f, 0.f, 0.f));
	CameraBoom->bDoCollisionTest = false; // Dont want to pull camera in when it collides with level

	// Create a camera...
	TopDownCameraComponent = CreateDefaultSubobject<UCameraComponent>(TEXT("TopDownCamera"));
	TopDownCameraComponent->SetupAttachment(CameraBoom, USpringArmComponent::SocketName);
	TopDownCameraComponent->bUsePawnControlRotation = false; // Camera does not rotate relative to arm

	// Create capture component + texture
	_textureRender = NewObject<UTextureRenderTarget2D>(this, TEXT("ScreenShot"));
	_textureRender->InitAutoFormat(512, 512);
	_textureRender->ClearColor = FColor::Magenta;
	_textureRender->bGPUSharedFlag = true;
	_textureRender->TargetGamma = 1.0f;
	_textureRender->UpdateResourceImmediate();

	Capture = CreateDefaultSubobject<USceneCaptureComponent2D>(TEXT("SceneCapture"));
	Capture->TextureTarget      = _textureRender;
	Capture->CaptureSource				  = SCS_FinalColorLDR;
	Capture->SetupAttachment(CameraBoom);

	// Activate ticking in order to update the cursor every frame.
	PrimaryActorTick.bCanEverTick = true;
	PrimaryActorTick.bStartWithTickEnabled = true;
}

void APaysanPawn::Tick(float DeltaSeconds)
{
    Super::Tick(DeltaSeconds);
}

void APaysanPawn::captureScene() const
{
    check(Capture != nullptr && _textureRender != nullptr);

	Capture->CaptureScene();
}

TArray64<uint8> APaysanPawn::dumpCapture() const
{
    check(Capture != nullptr && _textureRender != nullptr);

    TArray64<uint8> CompressedBitmap;
    FImage image;
    if (FImageUtils::GetRenderTargetImage(_textureRender, image))
    {
        if (!FImageUtils::CompressImage(CompressedBitmap, TEXT("jpg"), image))
        {
            verifyf(false, TEXT("Can't compress render target image to jpg"))
        }
    }
    else
    {
        verifyf(false, TEXT("Can't read render target image"));
    }

    return CompressedBitmap;
}

with this .h

UCLASS(Blueprintable)
class APaysanPawn : public APawn
{
	GENERATED_BODY()

public:
	APaysanPawn();

	// Called every frame.
	virtual void Tick(float DeltaSeconds) override;

	/** Returns TopDownCameraComponent subobject **/
	FORCEINLINE class UCameraComponent* GetTopDownCameraComponent() const { return TopDownCameraComponent; }
	/** Returns CameraBoom subobject **/
	FORCEINLINE class USpringArmComponent* GetCameraBoom() const { return CameraBoom; }

	void captureScene() const;
	TArray64<uint8> dumpCapture() const;

private:
	/** Top down camera */
	UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = Camera, meta = (AllowPrivateAccess = "true"))
	class UCameraComponent* TopDownCameraComponent = nullptr;

	/** Camera boom positioning the camera above the character */
	UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = Camera, meta = (AllowPrivateAccess = "true"))
	class USpringArmComponent* CameraBoom = nullptr;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = Camera, meta = (AllowPrivateAccess = "true"))
    class USphereComponent* SphereCapsule = nullptr;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = Camera, meta = (AllowPrivateAccess = "true"))
    class UNavMovementComponent* NavMovement = nullptr;

	UPROPERTY()
	UTextureRenderTarget2D*		_textureRender = nullptr;

    UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = Camera, meta = (AllowPrivateAccess = "true"))
    USceneCaptureComponent2D*	Capture = nullptr;
};

[RecourseDesign]

Hi Rominitch,

This is how I get the data from a rendertarget: (rtrgt is the rendertarget2d)

	TArray<FColor> data;
	FReadSurfaceDataFlags readFlags(RCM_UNorm); // between 0 and 1 RCM_SNorm is -1 - 1
	FTextureRenderTarget2DResource* rtrgtResource=(FTextureRenderTarget2DResource*)rtrgt->GetResource();
	rtrgtResource->ReadPixels(data,readFlags);

[Rominitch1 (Rominitch)]

I have still the same issue and no error on API.

[RecourseDesign]

It must be the scenecapture then - that looks pretty much the same as I’ve done in the past - the only thing I can think of is adding _textureRender->UpdateResourceImmediate(); before GetResource()…

[Rominitch1 (Rominitch)]

I try it but without sucess (with or without clearColor flag).

I show you my caller code.

UCLASS(Abstract, Blueprintable)
class UBaseGameInterface : public UUserWidget
{
  ...
   UFUNCTION()
   void onMenu();

   void NativeConstruct() override;

   UPROPERTY(BlueprintReadWrite, meta = (BindWidget))
   UButton*      menuButton = nullptr;
 ...
};

void UBaseGameInterface ::NativeConstruct()
{
   ...
   menuButton->OnClicked.AddDynamic(this, &UBaseGameInterface::onMenu);
   ...
}

void UBaseGameInterface::onMenu()
{
    check(_controller != nullptr);

    auto camera = Cast<APaysanPawn>(_controller->GetPawn());
    check(camera != nullptr);
    camera->captureScene();

    camera->dumpCapture();
}

EDIT
I try to see where is my issue so I wrote this code and still have zero buffer where i expect magenta …

TArray64<uint8> readTextureCleanColor()
{
	// Allocated local Texture Render
    auto textureRender = NewObject<UTextureRenderTarget2D>();
	check(textureRender != nullptr);
    textureRender->InitCustomFormat(512, 512, PF_B8G8R8A8, true);
    textureRender->ClearColor = FColor::Magenta;
    textureRender->UpdateResourceImmediate();

	// Create final buffer
    TArray<FColor> data;
    //data.AddUninitialized(textureRender->SizeX * textureRender->SizeY);

	// Method 1: Using readPixel
    FReadSurfaceDataFlags readFlags(RCM_UNorm); // between 0 and 1 RCM_SNorm is -1 - 1
    auto resourceTexture = textureRender->GetResource();
    check(resourceTexture);
    auto rtrgtResource = (FTextureRenderTarget2DResource*)resourceTexture;
    check(rtrgtResource != nullptr);
    if (!rtrgtResource->ReadPixels(data, readFlags))
    {
        verifyf(false, TEXT("Can't ReadPixels()"));
    }

	// Method 2: Using FImageUtils features
    TArray64<uint8> CompressedBitmap;
    FImage image;
    if (FImageUtils::GetRenderTargetImage(textureRender, image))
    {
        if (!FImageUtils::CompressImage(CompressedBitmap, TEXT("jpg"), image))
        {
            verifyf(false, TEXT("Can't compress render target image to jpg"));
        }
    }
    else
    {
        verifyf(false, TEXT("Can't read render target image"));
    }

    return CompressedBitmap;
}

[RecourseDesign]

Oh, I think I see the problem.

In your constructor you’re correctly creating the Capture with “CreateDefaultSubobject()” but you’re creating the _textureRender with NewObject<> which you shouldn’t do in a constructor.

If you move the creation of _textureRender and Capture to your Begin Play, and create the Capture with Capture=NewObject<USceneCaptureComponent2D>(this,USceneCaptureComponent2D::StaticClass()); instead of the CreateDefaultSubobject it will hopefully work.

[Rominitch1 (Rominitch)]

Thanks !!!

It was a little more complicated but indeed NewObject effects were inexplicable. I use Kismet API helper to manipulate RenderTarget. (Thanks to this tutorial to show this api Render Targets for Shaders without the Render Target Camera Overhead).

APaysanPawn::APaysanPawn()
{
        ...
        // Allocated capture WITHOUT RenderTarget
        Capture = CreateDefaultSubobject<USceneCaptureComponent2D> (TEXT("SceneCapture"));
	Capture->CaptureSource = SCS_FinalColorLDR;
	Capture->SetupAttachment(CameraBoom);
        ...
}

void APaysanPawn::OnConstruction(const FTransform& Transform)
{
	Super::OnConstruction(Transform);

        // Build RenderTarget
	_textureRender = UKismetRenderingLibrary::CreateRenderTarget2D(GetWorld(), 512, 512, RTF_RGBA8, FColor::Magenta);
	check(_textureRender);
	_textureRender->UpdateResourceImmediate();

        // Set to capture system
	Capture->TextureTarget      = _textureRender;
}

TArray64<uint8> APaysanPawn::dumpCapture() const
{
    check(Capture != nullptr && _textureRender != nullptr);

	_textureRender->UpdateResourceImmediate(false);

    TArray64<uint8> CompressedBitmap;
    FImage image;
    if (FImageUtils::GetRenderTargetImage(_textureRender, image))
    {
        if (!FImageUtils::CompressImage(CompressedBitmap, TEXT("jpg"), image))
        {
            verifyf(false, TEXT("Can't compress render target image to jpg"));
        }
    }
    else
    {
        verifyf(false, TEXT("Can't read render target image"));
    }

    return CompressedBitmap;
}

And the photo finish