How to update a UTexture object in C++ code correctly?

martinortiz · July 3, 2023, 5:08am

// THESE ARE GLOBAL TO THE CLASS
// IN MY HEADER FILE
	UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Webcamera")
		class UTexture2D* WebcameraTexture;
	UTexture2D* Tex01;
	UTexture2D* Tex02;
// ======================================

// Called every frame
void AWebCam::Tick(float DeltaTime)
{
	Super::Tick(DeltaTime);
	return;
	if ((x++ % (30 * 5)) == 0)
	{
		if (ImageSwitch)
			WebcameraTexture = Tex01;
		else
			WebcameraTexture = Tex02;

		ImageSwitch = !ImageSwitch;
	}
}

UE 5.2
C++ class in a Blueprint project.
(above code) I know I have the “plumbing” right in Unreal Engine blueprint…with dynamic instance material and texture parameter etc…
Because if I have
Tex01 = cat image
Tex02 = dog image…
My Tick() function will swap back and forth between dog and cat textures and it will update the material correctly when playing the “level”.

Now I have a directshow graph, and I have a capture buffer function I set to function in below code. It is a pass through function. I can just do nothing and return and the video will actually play…OUTSIDE of UE application. I only mention it, because that is 100% working fine. But of course the goal is to capture the video frames and “play” it on a texture. (yes, i know there are plugins that do this already).

I can change the video frame to be all blue, and it will display OUTSIDE of app as a blue screen video…

OK here is the capture frame function. Now one thing I noticed is that this is apparently called in the same thread the Tick() function gets called in so that I can do this with no problems…no need to lock texture…(NOTE this is not the Tick() function but my callback function. It works with no problems…exactly like Tick() version. Both TIck and Sample functions update the texture of material inside of Unreal app…not outside app.

void AWebCam::SampleCB(double SampleTime, IMediaSample* pSample)
{
	if ((x++ % (30 * 5)) == 0)
	{
		if (ImageSwitch)
			WebcameraTexture = Tex01;
		else
			WebcameraTexture = Tex02;

		ImageSwitch = !ImageSwitch;
	}
}

So here is actual function…I want to “play” a blue texture onto material inside of Unreal app. When that works, I will of course replace it with the actual image data from the video. This is code I grabbed from a dozen or more examples of the same thing…

void AWebCam::SampleCB(double SampleTime, IMediaSample* pSample)
{
	HRESULT hr;
	unsigned char* Buffer;
	hr = pSample->GetPointer((BYTE**)&Buffer);  // ACTUAL VIDEO FRAME (NOT USED YET)
	if (hr != S_OK)
		return;

               // FILL BRGA BUFFER WITH BLUE COLOR
		int32 w = Width;
		int32 h = Height;
		TArray<FColor> rawData;
		for (int32 i = 0; i < (w * h); i++)
			rawData.Add(FColor(0xFF, 0x00, 0x00, 0xFF));

               // MAKE LOCAL TEXTURE OBJECT, LOCK IT
		UTexture2D* texture = UTexture2D::CreateTransient(w, h);
		FTexture2DMipMap& Mip = texture->PlatformData->Mips[0];
		void* Data = Mip.BulkData.Lock(LOCK_READ_WRITE);

                // BLIT THE BLUE DATA INTO TEXTURE
		FMemory::Memcpy(Data, rawData.GetData(), (w * h * 4));

                // UNLOCK 
     		Mip.BulkData.Unlock();

		//texture->UpdateResource();  // IF I UNCOMMENT THIS, IT THROWS AN EXCEPTION.
		//WebcameraTexture = texture; // IF I UNCOMMENT THIS...NOTHING GETS UPDATED (IN THIS CASE TURNS BLUE
}

I have seen a dozen examples that are essentially the thing and done this way.
For some reason it is not working. What is wrong? Also, since I am running in same thread as Tick() function(called at different times of course), do I need to Lock anything at all?

martinortiz · July 3, 2023, 5:20am

		texture->UpdateResource();

If I uncomment above line I get an exception…why?
I made “texture” global to class instead of local variable, and I still get an exception on this line. (the exception does not provide any details)
The variable is not NULL.

		WebcameraTexture = texture;

If I uncomment above line, nothing really happens.
I don’t get a blue texture “playing” onto material inside of Unreal application.

However if I copy the Tick code, into my capture buffer function, the texture gets updated inside of Unreal application inside the level. With no locking needed.

martinortiz · July 3, 2023, 5:49am

Faked “video” of cat and dog images “playing” onto texture inside of UE5.2

martinortiz · July 3, 2023, 5:57am

Video “working” but playing outside of Unreal application.

martinortiz · July 3, 2023, 3:18pm

I was surprised to find out that the Tick function and my video callback handler, sample grabber function, both occur in the same thread. So my guess is that DirectShow infrastructure does the bulk of its work in its own thread, but calls the callback handler function in the thread that the callback function handler was assigned in.
But this begs the question: Is Locking read/write operations necessary?
In the UE Tick function, nothing needs to be locked.
In my CBH function, I think(?) that locking would not be necessary either???

martinortiz · July 3, 2023, 3:24pm

OOops…my mistake, I should have looked more closely.
The Tick function and my call back handler function are indeed in separate threads.
I thought they were both called in same threads.

martinortiz · July 3, 2023, 3:32pm

Quick question…this line here…

		UTexture2D* texture = UTexture2D::CreateTransient(w, h);

Do I need to do this every time in my CBH (call back handler)?
Can’t I just do this once in the constructor? Or somewhere?
Should it be a local variable inside CBH?
Or should it be a global to class level variable?
(the danger in making it global is that some code might touch the object in another thread)

martinortiz · July 3, 2023, 6:34pm

If I do this, it will work 100%. BUT ONLY IN SAME THREAD THAT TICK FUNCTION is in.
(that is, I think it’s named the game thread)
The exact code will not work in a different thread.

This code is more or less the same exact code I did in CBH function.
i end up with a texture changed to all red.

// int32 w = Width;
// int32 h = Height;
// NOT THIS -> UTexture2D* tmpTex = UTexture2D::CreateTransient(w, h);

// USE EXISTING TEXTURE IMAGE
UTexture2D* tmpTex = Cast(StaticLoadObject(UTexture2D::StaticClass(), NULL, TEXT(“/Script/Engine.Texture2D’/Game/WebCam/Images/TEMPTEX.TEMPTEX’”)));

//	changne all Pixels to RED
int32 Row = tmpTex->GetSizeY();
int32 Coloum = tmpTex->GetSizeX();
FColor* MipData = static_cast<FColor*>(tmpTex->PlatformData->Mips[0].BulkData.Lock(LOCK_READ_WRITE));

for (int32 RowNum = 0; RowNum < Row; ++RowNum)
{
	for (int32 ColNum = 0; ColNum < Coloum; ++ColNum)
	{
		FColor& CurColor = MipData[ColNum + (RowNum * Coloum)];
		//	change color
		CurColor = FColor(255, 0, 0, 255);
	}
}

tmpTex->PlatformData->Mips[0].BulkData.Unlock();

//	update data
tmpTex->UpdateResource();

martinortiz · July 3, 2023, 6:44pm

OK, so my issue has nothing to do with the texture updating code.
It has to do with threading…
If I do my code or anybody else’s code in main thread, it will work.
But I will get exceptions if I try to do the same thing in a separate thread.
If I use code that does not throw an exception, nothing gets updated or modified.

So what is the solution???