How to create tight 2d bounding boxes for actors in player's view

Development Programming & Scripting
1

Hi,

I need to get tight bounding boxes around actors in the player’s view (2D). My current method is 1. getting capsule component bounding box in 3D. AND 2. project 8 vertices into 2D and calculate the min&max.

However, this box is too wide in 2D due to the projection . For example, the green box in the image below is the capsule component bounding box in 3D and it’s tight. When projecting it to 2D, I get red box, which is too wide and the blue box is the box I want. Does anyone know how I can get the tight 2D bounding box?

272358-capture.png

2

I’m still trying to figure out bounding boxes in UE4 myself, but maybe we can help each other out in some way. I’m sure since you have some sort of bounds for your actor working you at least know how to set them up. How exactly do you setup up the current bounding boxes that you have already?

3

I already solved this by calculating bbox of each physical assets component. Due to license issue I can’t release the code here. The box is not minimum due to physical assets are not perfectly aligned with real shape but it’s good enogh. If you try this way, please be careful on the coordinate transformation!

4

The main problem is that when looking from different angles, the 3D bounding box looks a lot larger than the actual actor inside. A possible solution that I found, although not perfect, gets a much closer 2D box. Using a FOrientedBox object and set Center, Extents and Axis members to match your capsule’s bounding box, then execute CalcVertices on the FOrientedBox object, that will calculate the 8 vertices of the bounding box as seen from that angle, and then project those to the screen. The function is an adapted version from this thread, and outputs the origin and size of the resulting 2D box.

FOrientedBox oBox;
oBox.Center = Origin;
oBox.ExtentX = Extent.X;
oBox.ExtentY = Extent.Y;
oBox.ExtentZ = Extent.Z;
oBox.AxisX = AxisX;
oBox.AxisY = AxisY;
oBox.AxisZ = AxisZ;

FVector OutVertices[8];
oBox.CalcVertices(OutVertices);


const FVector BoundsPointMapping[8] =
{
	FVector(1, 1, 1),
	FVector(1, 1, -1),
	FVector(1, -1, 1),
	FVector(1, -1, -1),
	FVector(-1, 1, 1),
	FVector(-1, 1, -1),
	FVector(-1, -1, 1),
	FVector(-1, -1, -1)
};

// Build 2D bounding box of actor in screen space
FBox2D ActorBox2D(0);
for (uint8 BoundsPointItr = 0; BoundsPointItr < 8; BoundsPointItr++)
{
	FVector2D ScreenLocation;
	
	// Project vert into screen space.
	controller->ProjectWorldLocationToScreen(OutVertices[BoundsPointItr], ScreenLocation);
	// Add to 2D bounding box
	ActorBox2D += ScreenLocation;// FVector2D(ProjectedWorldLocation.X, ProjectedWorldLocation.Y);
}
outOrigin = ActorBox2D.Min;
outSize = ActorBox2D.GetSize();
1 Like
5

I’m having the same problem, is there maybe any better solution now with UE5?

1 Like
6

@anonymous_user_78291dd0 any chance you can share your code now?

7

This plugin does the trick:

8

You can get all the vertices and normals from mesh and calculate it from those. Though I assume there is a much more efficient method exists.
Somewhere in Init method or BeginPlay:

void BoundingBoxActor::Init(AActor* InActor)
{
        Actor = InActor;
		PC = Actor->GetWorld()->GetFirstPlayerController();
		if (TObjectPtr<UStaticMesh> Mesh = Actor->GetComponentByClass<UStaticMeshComponent>()->GetStaticMesh(); Mesh->bAllowCPUAccess)
		{
			TArray<int32> OutTriangles;
			TArray<FVector2D> OutUVs;
			TArray<FVector> OutNormals;
			TArray<FVector> OutVertices;
			TArray<FProcMeshTangent> OutTangents;
			for (int s = 0; s < Mesh->GetNumSections(0); ++s)
			{
				UKismetProceduralMeshLibrary::GetSectionFromStaticMesh(Mesh, 0, s, OutVertices, OutTriangles, OutNormals, OutUVs, OutTangents);
				for (int i = 0; i < OutNormals.Num(); ++i)
				{
					if (Normals.Find(OutNormals[i]) == INDEX_NONE)
					{
						Normals.Emplace(OutNormals[i]);
						Vertices.Emplace(OutVertices[i]);
					}
				}
			}
		}
}

Then in Tick or when you need to update BBox you can do the following:

            FBox2D BBox;

			FVector Forward = PC->PlayerCameraManager->GetActorForwardVector();
			for (int i = 0; i < Normals.Num(); ++i)
			{
				if (Forward.Dot(Normals[i]) < 0)
				{
					UGameplayStatics::ProjectWorldToScreen(PC, T.TransformPosition(Vertices[i]), ScreenPosition);

					BBox += ScreenPosition;
				}
			}

BBox will be basically Rect in Screen coordinates (top left cornet/bottom right corner). And you can draw lines in UI from those.