ProjectWorldLocationToScreen

rubm123 · September 1, 2022, 5:16pm

I am implementing a screen indicator in the HUD. It always points into the direction of the Actor called F1 to give the player an idea what direction to go to. This is no groundbreaking stuff. In theory, this is how I do it:

In the tick method of the HUD:

Step 1: Get the screen location of F1

FVector2D ScreenLocation;
const bool bProjected = GetOwningPlayerController()->ProjectWorldLocationToScreen(VecF1, ScreenLocation, true);

Step 2A: Convert to coordinates that have the center of the screen as origin:

const float XFromCenter = ScreenLocation.X - .5;
const float YFromCenter = ScreenLocation.Y - .5;

Note: Alternatively, you can use ProjectWorldLocationToScreen with false (the default value of the last parameter) and then do the conversion to relative coordinates manually like this:

Step 2B: Manually convert to relative coordinates

const auto Vec2DSize = UWidgetLayoutLibrary::GetViewportSize(GetWorld());
const float XFromCenter = ScreenLocation.X / Vec2DSize.X - .5;
const float YFromCenter = ScreenLocation.Y / Vec2DSize.Y - .5;

… only that 2A and 2B have different results. I don’t understand what that last parameter means.

Step 3: Check if F1 is off screen:

if(abs(XFromCenter) > 0.5 || abs(YFromCenter) > 0.5)
{
    // ...
}

Step 4: Check if F1 is above/below the viewport: the “vertical case”, in constrast to to the left of/to the right of the viewport (the “horizontal case”):

if(abs(YFromCenter / XFromCenter) > 1.)
{
    // above/below viewport, "vertical case"
    // ...
}

Step 5: Get the x screen location for the indicator overlay

OverlayX = XFromCenter * 0.5 / abs(YFromCenter);

Step 6: Get the y screen location for the indicator overlay:

OverlayY = XFromCenter < 0 ? -0.5 : 0.5;

Step 7: Do the same for the horizontal case

Step 8: Set the position of the indicator overlay:

const auto ViewportScale = UWidgetLayoutLibrary::GetViewportScale(GetWorld());	
UWidgetLayoutLibrary::SlotAsCanvasSlot(CanvasF1)->SetPosition(FVector2D((OverlayX + .5) * Vec2DSize.X, (OverlayY + .5) * Vec2DSize.Y) / ViewportScale);

There are quite a number of potential mistakes.

Conversion with ViewportScale
Conversion to relative coordinates
Calculation of OverlayX and OverlayY

But I have high confidence in that part of the code. I am frustrated by the fact that ProjectWorldLocationToScreen apparently isn’t well behaved. Sometimes it returns false, indicating that the world location couldn’t be projected. But when is that the case? When the actor is behind the camera, maybe? I don’t get it.

Checking my code, ProjectWorldLocationToScreen returns false quite erratically. I have a tilted top-down view on my character and F1 is always on the plane that I am looking down at … given that, I might implement my own version of ProjectWorldLocationToScreen, but how odd. Isn’t my use case very straightforward?

Then, the two versions of ProjectWorldLocationToScreen, differentiated by the last parameter: Why do my alternatives have different outcomes? What does that parameter do?

EDIT: I fixed one bug in my code and removed the question that corresponded to it.

Shmoopy1701 · September 1, 2022, 5:40pm

Yeah, there’s a lot of places where that code can go wrong. We have a very similar use case and are using “UWidgetLayoutLibrary::ProjectWorldLocationToWidgetPosition()” so maybe look if that produces different results?

rubm123 · September 1, 2022, 6:25pm

UWidgetLayoutLibrary::ProjectWorldLocationToWidgetPosition() relies on UWidgetLayoutLibrary::ProjectWorldLocationToScreenWithDistance under the hood and then adds some calculations from “screen coordinates” to “viewport” coordinates that I don’t fully understand.

Those conversion might be necessary to properly adjust for different screen resolutions and aspect ratios.

rubm123 · September 14, 2022, 5:33pm

So I was able to fix my code.

Indeed it seems that ProjectWorldLocationToScreen only works (and returns true) when the object is in front of the camera. Note that “in front of” includes the case that the object is off screen. When the object is behind the camera and inside the mirrored view-cone the projection to screen would result in on-screen-coordinates. Apparently the developers preferred to return false and no coordinates instead.

If that could be confirmed somehow, or documented even, I would be so happy.

My solution is this: When the projection fails, I manually calculate an object position in front of the camera and project that to the screen and then I translate the screen position to the corresponding edge of the screen.

This last step I was missing and it was confusing me to no ends.

And I still don’t understand the last parameter of ProjectWorldLocationToScreen. About ProjectWorldLocationToWidgetPosition I only see the more complicated code that make it seem less desirable performance-wise. I might try it out later, anyway.

suren4crazy · November 9, 2022, 6:41am

Hi, did you achieve the solution? it would be helpful if you share, how you done that.

rubm123 · November 9, 2022, 4:45pm

gist.github.com

https://gist.github.com/rubenmoor/d4afad7de45fd206ad07417845009570

MyHUD.cpp

void AMyHUD::Tick(float DeltaSeconds)
{
	Super::Tick(DeltaSeconds);
	
  const APlayerController* PC = GetOwningPlayerController();
  const FVector2D Vec2DSize = UWidgetLayoutLibrary::GetViewportSize(GetWorld());
	FVector2D ScreenLocation;
	
  // `FVector VecF1` is the location of F1, the point in world space that I am tracking via screen indicator 
	// try to project to screen coordinates, ...

This file has been truncated. show original

suren4crazy · November 10, 2022, 5:37am

Thank you

rubm123 · November 10, 2022, 12:38pm

I added some more extensive information on the whole topic here:

github.com

rubenmoor/projectzero/blob/main/TrackingIndicator.md

# Recipe: Tracking indicator for quest marks or important objects in the player's world

Imagine there is an important object in the world that you want to highlight for the player.
When the object is not visible on the screen you want to show an arrow at the edge of the screen, pointing in the direction of the objection.
In more technical terms:
If the projection of the objects world location to screen coordinates results in a position outside the viewporty, you display an indicator.

This mechanism makes sense for both, first-person view and third-person view.
The below example assumes a third-person view.

## The indicator

I use the [bindwidget approach](https://benui.ca/unreal/ui-bindwidget/) to access HUD elements in C++.
This means, I create a C++ class, `UW_HUD`, that inherits from `UUserWidget` and has the following member:

```cpp
UPROPERTY(meta=(BindWidget))
TObjectPtr<UCanvasPanel> CanvasIndicator;

UPROPERTY(meta=(BindWidget))

This file has been truncated. show original

anonymous_user_7b0094b91 · August 13, 2024, 2:09pm

Hi,

Is there any chance that your projections are delayed by one tick?

We’re fighting this problem that all our projections to 2D can’t be resolved the moment we programmatically move the camera, because we always get a projection based on the previous position.

It would be lovely to hear from you if you feel that your projections are delayed by a frame, giving a rubbery feel to the HUDs when actors change direction quickly.

Per