USceneCaptureComponent2D to Slate SImage

KarolRoesner · July 5, 2024, 1:25pm

Hey Everyone,
i am trying to write a Plugin in UnrealEngine 5.4.
I started with an “Editor Standalone Window” template.

I want to be able to display the Image of an Camera inside of this Window.
My current progress looks like this.

Creating a new CameraClass from ACameraActor with USceneCaptureComponent2D* to capture the image.

//.h File
#pragma once

#include "CoreMinimal.h"
#include "Camera/CameraActor.h"
#include "Components/SceneCaptureComponent2D.h"
#include "Engine/TextureRenderTarget2D.h"
#include "OmniTestCameraActor.generated.h"

UCLASS()
class RENDERCAMERA_API AOmniTestCameraActor : public ACameraActor
{
	GENERATED_BODY()

public:
	AOmniTestCameraActor();

	UPROPERTY(VisibleAnywhere)
	USceneCaptureComponent2D* SceneCaptureComponent;
};

//.cpp File
#include "OmniTestCameraActor.h"

AOmniTestCameraActor::AOmniTestCameraActor()
{
	SceneCaptureComponent = CreateDefaultSubobject<USceneCaptureComponent2D>(TEXT("SceneCaptureComponent"));
	SceneCaptureComponent->SetupAttachment(RootComponent);

	SceneCaptureComponent->TextureTarget = CreateDefaultSubobject<UTextureRenderTarget2D>(TEXT("RenderTarget"));
	SceneCaptureComponent->TextureTarget->InitAutoFormat(1024, 1024);
}

A class SMyCompoundWidget with a FSlateBrush and a UTextureRenderTarget2D.

#pragma once

#include "CoreMinimal.h"
#include "Widgets/SCompoundWidget.h"
#include "Engine/TextureRenderTarget2D.h"
#include "Brushes/SlateDynamicImageBrush.h"
#include "Slate/DeferredCleanupSlateBrush.h"

class RENDERCAMERA_API SMyCompoundWidget : public SCompoundWidget
{
public:
	SLATE_BEGIN_ARGS(SMyCompoundWidget){}
	SLATE_END_ARGS()

	/** Constructs this widget with InArgs */
	void Construct(const FArguments& InArgs, UTextureRenderTarget2D* InRenderTarget);
	
	TSharedPtr<FSlateBrush> Brush; 

private:
	UTextureRenderTarget2D* RenderTarget = nullptr;
};

//.cpp File
#include "SMyCompoundWidget.h"
#include "SlateOptMacros.h"
#include "Slate/DeferredCleanupSlateBrush.h"

BEGIN_SLATE_FUNCTION_BUILD_OPTIMIZATION

void SMyCompoundWidget::Construct(const FArguments& InArgs, UTextureRenderTarget2D* InRenderTarget)
{
	if(InRenderTarget)
	{
		Brush = MakeShareable(new FSlateBrush());
		Brush->SetResourceObject(InRenderTarget->ConstructTexture2D(InRenderTarget, "TextureCamera", RF_NoFlags));
		Brush->SetImageSize(FVector2D(256,256));
	}
	
	ChildSlot
	[
		SNew(SImage).Image(Brush.Get())
	];
}

So from my understanding, I cant get an image directly from a camera. I have to create a SceneCaptureComponent2D, set the TextureTarget and then use this with a FSlateBrush to paint it inside an Image. My problem now is that i am very new in UnrealEngine and have big problems to understand how everything works in details. My end result generates a Camera, this camera does have a USceneCaptureComponent2D and the Texture Target of that seems to exist. But my window shows only a black screen.

My ideas of what is wrong are the following:

The Brush does work, but it gives the SImage only one image and that image is no existing during the generation of everything. Would mean i have to update the SImage all the time?
No clue how to realize that.
I am using the Brush in a completely wrong way.

Here is the OnSpawnPluginTab Function to spawn everything.

TSharedRef<SDockTab> FRenderCameraModule::OnSpawnPluginTab(const FSpawnTabArgs& SpawnTabArgs)
{
	FText WidgetText = FText::Format(
		LOCTEXT("WindowWidgetText", "Add code to {0} in {1} to override this window's contents"),
		FText::FromString(TEXT("FRenderCameraModule::OnSpawnPluginTab")),
		FText::FromString(TEXT("RenderCamera.cpp"))
		);
	
	if (const UWorld* World = GEditor->GetEditorWorldContext().World())
	{
		ULevel* Level = World->GetCurrentLevel();
		if (AActor* NewActor = GEditor->AddActor(Level, AOmniTestCameraActor::StaticClass(), FTransform::Identity))
		{
			NewActor->SetActorLabel("OmniCameraActor");
			OmniCameraActor = Cast<AOmniTestCameraActor>(NewActor);
		}
	}
	
	return SNew(SDockTab)
		.TabRole(ETabRole::NomadTab)
		[
			// Put your tab content here!
			SNew(SBox)
			.HAlign(HAlign_Center)
			.VAlign(VAlign_Center)
			[
				SNew(SMyCompoundWidget, this->OmniCameraActor->SceneCaptureComponent->TextureTarget)
			]
		];
}

I am greatful for any help, any recomendation for better code style and any explanation on what to do.