How to render dynamic Images with slate (In-Editor)

Hello pals and gals,

I have image data (TArray<FColor>) which gets updated almost every frame from inside the Tick Event of an actor, that is placed in the level.

Now I want to display this image in the editor. How do I do that? I have already created a new editor window and I know how to render material and texture assets. But I don’t get how to render image data, that is created dynamically.

I know this sounds like I am asking for a complete solution, but I am greateful about anyone who is able to push me in the right direction. I have already tried and tested a lot of stuff, but it’s so much that I decided to elaborate on it when someone throws in an idea that is similar. For now, I am appealing to the collective knowledge of all UE users :smiley:

Regards,

hi, SImage or other are receiving FSlateBrush, every brush receive a texture, then just plug your texture there, actually the entire solution is there for you, here is a wiki that some kind lord wrote some time ago: A new, community-hosted Unreal Engine Wiki - Announcements and Releases - Unreal Engine Forums

Thanks a lot ZkarmaKun, I already tried that way and unfortunately, the source above is partly out-of-date. I actually had to give up on rendering a texture and finally found a way to render the image by using a dynamic material instance late last night!
I will post a tutorial for this when I find the time.
But when I do, I would still like to figure out why I couldnt make a brush from a UTexture2D :confused:

hi rYuxq, give it a try again, that method is working perfectly for me on 4.18, perhaps you are doing something off

You are actually right. But I have difficulties in getting a reference to the UTexture2D from where I spawn my window tab. When I press a button on the toolbar (above the ViewPort), a certain function is triggered, called


TSharedRef<SDockTab> FMyEditorModule::OnSpawnPluginTab(const FSpawnTabArgs& SpawnTabArgs)

If you create a new plugin using the “Editor Standalone Window” template, you will get a class that has exactly this function signature and you will see where I’m coming from.

I am currently accessing it by using an ObjectIteator. But even if I get the right reference, as soon as I open my editor tab and then quit the play-in-editor game, I get a crash. I suspect that that the problem is that the SImage is using a resource (my UMaterialInstanceDynamic or UTexture2D) that doesn’t exist anymore, since it had been created at runtime.
The strange thing is, in the case where I use UTexture2D, the engine doesn’t crash when I quit the game, but when I restart the game after quitting it once.I think this has something to do with me assigning the UTexture2D a (unique) FName… (at least that’s what the crash message indicates)

The only way to avoid any crashes is for me to find the actor through the Object Iterator and then setting the texture by using his property.

Here is a minimal implementation (of my best approach so far) of the function mentioned above:


TSharedRef<SDockTab> FMyEditorModule::OnSpawnPluginTab(const FSpawnTabArgs& SpawnTabArgs)
{
    AMyActor* actor = nullptr;
    for (TObjectIterator<UTexture2D> Itr; Itr; ++Itr)
    {
            // find the right actor
            actor = *Itr;
    }

    this->Brush.SetResourceObject(actor->Texture);
    this->Brush.ImageSize = FVector2D(512, 512);

    return SNew(SDockTab)
        .TabRole(ETabRole::NomadTab)
        
            // Put your tab content here!
            SNew(SGridPanel)
            + SGridPanel::Slot(0, 0).HAlign(HAlign_Left).VAlign(VAlign_Top)
        
            SNew(SImage)
            .Image(&Brush)
        ]
        ];
}

I deleted some lines to simplify this, but I think this should work an compile.

When I create my UTexture2D at MyActor::BeginPlay(), I rename it to give it this unique name so that I can identify it when I spawn my plugin tab.
This doesn’t seem to be a good way to handle this… Also I dont know how to deal with this issue that seems to be related to garbage collection.

Ah, ok, I see your problem, yes that’s normal, every UObject including AActors are garbage collected, after you quit PIE, everybody will be clean, and you are crashing because you are using your texture as reference, you have multiple options here:
First one: clean your references before PIE ends, you can catch this binding a delegate for FEditorDelegates, this solution is not elegant at all :smiley:
Second option: make your texture global, like everything in this life you can spawn some UObject and make it eternal by calling AddToRoot(), who needs an actor, but remove it from root after in your code, otherwise it will be there forever. another not elegant solution :frowning:
Third one: use FSlateTexture2DRHIRef, this behave like every other texture, the only problem is implement your tick and update your texture there, you can do it inside a class, and this as a TSharedPtr<> will be ideal

Finally in slate you should use TWeakObjectPtr to address you actor references, using naked references from Garbage Collected classes is only if you are looking for trouble

hope that helps

Thank you, this sounds very useful.

So what I would like to do is to create a new UClass type (for editor only) that stores the reference to the texture and a reference to my SImage, so that I can ObjectIterate through all those types when PIE ends, allowing me to replace the brush of all SImages with a constant image.

But creating a constant brush from image data doesnt seem to be possible. At least it isnt obvious to me. There is FSlateImageBrush, but it needs an object, not byte data.

Do you know a way to show a constant image? Is my approach even reasonable at all?

hi rYuxq, yes, first try to see any Style class from the source code, take a look at FEditorStyle class, you will see how UE implements simple textures to use in menus or something else, FSlateImageBrush is able to read some image from drive and keep it as long as you want depending on your code.

this a fast example:



FSlateImageBrush* myBrush = new FSlateImageBrush("your full image path", FVector2D(128,128));


I’m guessing that you are filling your TArray<uint8> directly with an image from some path, FSlateImageBrush will do that for ya, otherwise you can initialize a texture an set your Source

Unfortunately thats not the case, this data is generated at runtime, dynamically. To be precise, I am processing the scene caught by a SceneCaptureComponent2D. Any ideas?

I would like to avoid creating a texture for a static image, but if this is really the only way, I will have to go with this…
Thanks a lot for your help, this is very helpful!

Just for the sake of completeness. How would I go about implementing this FSlateTexture2DRHIRef stuff? I actually knew about this struct beforehand but I had failed to utilize it…

I just checked the source code of FSlateImageBrush and it seems like they always use textures. So even png files seem to be loaded as textures. I will just do the same then
Thanks a lot!