How to create tight 2d bounding boxes for actors in player's view

Development Programming & Scripting
1

Hi,

I need to get tight bounding boxes around actors in the player’s view (2D). My current method is 1. getting capsule component bounding box in 3D. AND 2. project 8 vertices into 2D and calculate the min&max.

However, this box is too wide in 2D due to the projection . For example, the green box in the image below is the capsule component bounding box in 3D and it’s tight. When projecting it to 2D, I get red box, which is too wide and the blue box is the box I want. Does anyone know how I can get the tight 2D bounding box?

272358-capture.png

2

I’m still trying to figure out bounding boxes in UE4 myself, but maybe we can help each other out in some way. I’m sure since you have some sort of bounds for your actor working you at least know how to set them up. How exactly do you setup up the current bounding boxes that you have already?

3

I already solved this by calculating bbox of each physical assets component. Due to license issue I can’t release the code here. The box is not minimum due to physical assets are not perfectly aligned with real shape but it’s good enogh. If you try this way, please be careful on the coordinate transformation!

4

The main problem is that when looking from different angles, the 3D bounding box looks a lot larger than the actual actor inside. A possible solution that I found, although not perfect, gets a much closer 2D box. Using a FOrientedBox object and set Center, Extents and Axis members to match your capsule’s bounding box, then execute CalcVertices on the FOrientedBox object, that will calculate the 8 vertices of the bounding box as seen from that angle, and then project those to the screen. The function is an adapted version from this thread, and outputs the origin and size of the resulting 2D box.

FOrientedBox oBox;
oBox.Center = Origin;
oBox.ExtentX = Extent.X;
oBox.ExtentY = Extent.Y;
oBox.ExtentZ = Extent.Z;
oBox.AxisX = AxisX;
oBox.AxisY = AxisY;
oBox.AxisZ = AxisZ;

FVector OutVertices[8];
oBox.CalcVertices(OutVertices);


const FVector BoundsPointMapping[8] =
{
	FVector(1, 1, 1),
	FVector(1, 1, -1),
	FVector(1, -1, 1),
	FVector(1, -1, -1),
	FVector(-1, 1, 1),
	FVector(-1, 1, -1),
	FVector(-1, -1, 1),
	FVector(-1, -1, -1)
};

// Build 2D bounding box of actor in screen space
FBox2D ActorBox2D(0);
for (uint8 BoundsPointItr = 0; BoundsPointItr < 8; BoundsPointItr++)
{
	FVector2D ScreenLocation;
	
	// Project vert into screen space.
	controller->ProjectWorldLocationToScreen(OutVertices[BoundsPointItr], ScreenLocation);
	// Add to 2D bounding box
	ActorBox2D += ScreenLocation;// FVector2D(ProjectedWorldLocation.X, ProjectedWorldLocation.Y);
}
outOrigin = ActorBox2D.Min;
outSize = ActorBox2D.GetSize();
1 Like
5

I’m having the same problem, is there maybe any better solution now with UE5?

1 Like
6

@anonymous_user_78291dd0 any chance you can share your code now?

7

This plugin does the trick:

8

You can get all the vertices and normals from mesh and calculate it from those. Though I assume there is a much more efficient method exists.
Somewhere in Init method or BeginPlay:

void BoundingBoxActor::Init(AActor* InActor)
{
        Actor = InActor;
		PC = Actor->GetWorld()->GetFirstPlayerController();
		if (TObjectPtr<UStaticMesh> Mesh = Actor->GetComponentByClass<UStaticMeshComponent>()->GetStaticMesh(); Mesh->bAllowCPUAccess)
		{
			TArray<int32> OutTriangles;
			TArray<FVector2D> OutUVs;
			TArray<FVector> OutNormals;
			TArray<FVector> OutVertices;
			TArray<FProcMeshTangent> OutTangents;
			for (int s = 0; s < Mesh->GetNumSections(0); ++s)
			{
				UKismetProceduralMeshLibrary::GetSectionFromStaticMesh(Mesh, 0, s, OutVertices, OutTriangles, OutNormals, OutUVs, OutTangents);
				for (int i = 0; i < OutNormals.Num(); ++i)
				{
					if (Normals.Find(OutNormals[i]) == INDEX_NONE)
					{
						Normals.Emplace(OutNormals[i]);
						Vertices.Emplace(OutVertices[i]);
					}
				}
			}
		}
}

Then in Tick or when you need to update BBox you can do the following:

            FBox2D BBox;

			FVector Forward = PC->PlayerCameraManager->GetActorForwardVector();
			for (int i = 0; i < Normals.Num(); ++i)
			{
				if (Forward.Dot(Normals[i]) < 0)
				{
					UGameplayStatics::ProjectWorldToScreen(PC, T.TransformPosition(Vertices[i]), ScreenPosition);

					BBox += ScreenPosition;
				}
			}

BBox will be basically Rect in Screen coordinates (top left cornet/bottom right corner). And you can draw lines in UI from those.

9

hello，I purchased this plugin, but I encountered some problems while using it. After enabling the plugin, the components from the plugin did not appear. I copied these components into my project, but the display has been completely black.

10

Hello guys! I’m too late :sweat_smile:
I, too, encountered this problem: I created a 2D bounding box, but when I tried applying it to a sphere, the frame didn’t align correctly. So, I took a look at the game Voices of the Void; according to the developers, the entire game was built using Blueprints, and it features a similar 2D bounding frame that scales perfectly to fit spheres and all other objects—and, notably, it doesn’t appear to rely on standard bounding boxes. Consequently, I set out to create a similar frame myself. However, I ran into issues with the standard bounding box approach, so I wrote some C++ code that retrieves all the collision vertices of an object and places points along that collision mesh. I then adapted my original bounding box logic to work with this new setup, utilizing my custom GetPrimitiveVertices node. Yet, I hit another roadblock: if the object’s collision geometry is complex, launching the game causes my frame rate to plummet from 80 FPS down to 20.
Unfortunately, I still haven’t been able to find a solution. Maybe someone will notice me—that would be cool! :slight_smile: