How to get the screen location of a viewport?

Development Programming & Scripting
1

Hello everyone!

I’m currently implementing a plugin and I need to transform some coordinates in operating system screen space to the active viewport.
I’ve gotten as far as finding a reference to the viewport I’m interested in:
GEngine->GameViewport->Viewport

I’m guessing this will always resolve to the viewport in question (disregarding any potential situations where it might be null). The question then becomes; How do I transform my screen coordinates to local coordinates in this viewport?

I first approached the question from the mouse angle. Since the unreal engine somehow is able to transform the mouse coordinates (expressed in “operating system screen space”) to viewport space coordinates, I should be able to piggyback on that code to transform my coordinates… right?

Well it turned out to not be that easy. I think I’ve found the part of the code that does this, in SceneViewport.cpp:


void FSceneViewport::GetMousePos( FIntPoint& MousePosition, const bool bLocalPosition )
{
	if (bLocalPosition)
	{
		MousePosition = CachedMousePos;
	}
	else
	{
		const FVector2D AbsoluteMousePos = CachedGeometry.LocalToAbsolute(FVector2D(CachedMousePos.X, CachedMousePos.Y));
		MousePosition.X = AbsoluteMousePos.X;
		MousePosition.Y = AbsoluteMousePos.Y;
	}
}

This class is the actual type of GEngine->GameViewport->Viewport it seems.
The problem here is that CachedGeometry is private, and I can’t find a good way to get access to it.
I’m currently trying to find a way to intercept CachedGeometry before it reaches the SceneViewport class, but it’s proving to be hard.
I’ve tried a few hacks to subvert the encapsulation system to see if it could work, and it seems like it, but it’s hardly a good solution.

Does anyone know of a better solution, or of a good way to make this work?

UPDATE:

Rama suggested using the:

	/** Transforms a point from world-space to the view's screen-space. */
	FVector4 WorldToScreen(const FVector& WorldPoint) const;

	/** Transforms a point from the view's screen-space to world-space. */
	FVector ScreenToWorld(const FVector4& ScreenPoint) const;

	/** Transforms a point from the view's screen-space into pixel coordinates relative to the view's X,Y. */
	bool ScreenToPixel(const FVector4& ScreenPoint,FVector2D& OutPixelLocation) const;

	/** Transforms a point from pixel coordinates relative to the view's X,Y (left, top) into the view's screen-space. */
	FVector4 PixelToScreen(float X,float Y,float Z) const;

	/** Transforms a point from the view's world-space into pixel coordinates relative to the view's X,Y (left, top). */
	bool WorldToPixel(const FVector& WorldPoint,FVector2D& OutPixelLocation) const;

	/** Transforms a point from pixel coordinates relative to the view's X,Y (left, top) into the view's world-space. */
	FVector4 PixelToWorld(float X,float Y,float Z) const;

Class of functions on FSceneView, but the screen referenced here is just another coordinate system inside the viewport itself, and are not related to the O/S screen pixels.

Best regards,
Temaran

2

#SceneView Screen to Pixel

If you can get your FSceneView, then you want these two functions:

#SceneManagement.h

bool FSceneView::ScreenToPixel(const FVector4& ScreenPoint,FVector2D& OutPixelLocation) const

FVector4 FSceneView::PixelToScreen(float InX,float InY,float Z) const

#:heart:

Rama

3

Very nice, I’ll definately look into this!

Can’t believe I missed these two functions, I was grepping around for “ToScreen” etc, but I must have overlooked it.
Thank you Rama!

4

And if someone is wondering, the easiest way to get the sceneview is probably:


ULocalPlayer* LocalPlayer = Cast(Player);

	if (LocalPlayer != NULL && LocalPlayer->ViewportClient != NULL && LocalPlayer->ViewportClient->Viewport != NULL)
	{
		// Create a view family for the game viewport
		FSceneViewFamilyContext ViewFamily( FSceneViewFamily::ConstructionValues(
			LocalPlayer->ViewportClient->Viewport,
			GetWorld()->Scene,
			LocalPlayer->ViewportClient->EngineShowFlags )
			.SetRealtimeUpdate(true) );

		// Calculate a view where the player is to update the streaming from the players start location
		FVector ViewLocation;
		FRotator ViewRotation;
		FSceneView* SceneView = LocalPlayer->CalcSceneView( &ViewFamily, /*out*/ ViewLocation, /*out*/ ViewRotation, LocalPlayer->ViewportClient->Viewport );

		if (SceneView)
		{
			SceneView->ScreenToPixel(PROPERARGS);
		}
	}
5

After running a few tests, it seems that this wasn’t what I needed after all.
The screen referenced in these function names are just another coordinate system in the viewport it seems, not the actual physical screen.

My quest for a way to properly do this continues!

Thank you for the idea though :slight_smile:

6

I’m also very interested in the answer to this question, have you found out a solution?

7

I haven’t tried it myself, but I’m pretty sure you need FViewport::ViewportToVirtualDesktopPixel (FViewport::ViewportToVirtualDesktopPixel | Unreal Engine Documentation)

8

Haha, I’d completely forgotten about this thread. I was the one who added that function to the engine a couple of months ago, but ty anyways for taking the time :slight_smile: