How to use Custom Stencil Mask in C++ code for 2D bounding box?

Development Programming & Scripting C++
, , ,
1

Hello Everyone,

I am currently trying to create a synthetic dataset for a Image Recognition project. I am trying to create precise Bounding Box around a drone for my dataset labels.

I was able to create a mask of my drone by following this tutorial : How to Create Masks With the Custom Stencil Buffer | Tips and Tricks | Unreal Engine | Talks and demos

Here is the result I obtained:

image
image1920×1221 219 KB

image
image2074×1307 8.31 KB

Now, I would like to create a 2D bounding box, probably by following this piece of code:

FBox2D UTryPixelAccess::calcBoundingFromBinary(UTextureRenderTarget2D * RenderTexture)
{
	TArray<FLinearColor> ImageData;
	FRenderTarget *RenderTarget = RenderTexture->GameThread_GetRenderTargetResource();
	RenderTarget->ReadLinearColorPixels(ImageData);

	FVector2D maxPoint(0, 0);
	FVector2D minPoint(RenderTexture->SizeX, RenderTexture->SizeY);

	for (int x = 0; x < RenderTexture->SizeX; x++) {
		for (int y = 0; y < RenderTexture->SizeY; y++) {
			int i = x + y * RenderTexture->SizeX;
			if (ImageData[i].R > 0.1) {
				// we a have white pixel
				if (x <= minPoint.X) {
					minPoint.X = x;
				}
				if(y <= minPoint.Y) {
					minPoint.Y = y;
				}
				if (x >= maxPoint.X) {
					maxPoint.X = x;
				}
				if (y >= maxPoint.Y) {
					maxPoint.Y = y;
				}
			}
		}
	}

	return FBox2D(minPoint, maxPoint);
}

from this question: Bounding Box from specific View (SceneCapture)

However :

  • I don’t know how to convert my Custom Stencil Buffer to a UTextureRenderTarget2D
  • I would like to render my images using the Movie Render Queue and I don’t know how to render at the same time the lit scene and get the Custom Stencil Buffer to calculate by BB.

I don’t care about performances as I will do a render afterward.

Thanks for your help.
Louis