Deproject Screen To World and SceneCapture2D.

Development Programming & Scripting C++
1

Hi,

I have a SceneCapture2D actor capturing my scene, and the output is displayed in a widget. When the user click on the widget, I would like to know where she clicked in the 3D space.

Basically “Deproject Screen to World” in blueprint, but for a SceneCapture2D.

I found an old post about it, but the hack doesn’t work.

Any idea about how I could do that ?

2

Hey did you find a solution? I only found a solution if the camera has the same field of view and ratio than the player camera. I have just calculated the relative click location from the player camera and than added the SceneCapture2D transforms to it. It works but only when both cameras have the same settings… Could you find a general solution…?

3

Maybe this will help:
https://answers.unrealengine.com/questions/528282/linetracebychannel-and-gethitresultundercursorbych.html

4

I donated Blueprint functions to Rama’s Victory plugin that project/deproject second capture component 2d’s.
Provided they still work, its probably your best place to start.

5

Hey Kris, you said that you contributed SceneCaptureComponent2D Project/Deproject functions, but I only see the Project function. Right now I desperately need a Deproject function to be able to Line Trace from my SceneCaptureComponent2D when I click in a Widget which renders a RenderTarget texture. There are so many requests in AnswerHub but no solutions. Can you help me (and others) with that one?

6

Took a quick look on GitHub at Rama’s Victory Plugin and yeah, couldn’t see the Deproject functions.
Odd, coz, well, I swear I included them when I included Project :expressionless:

I am currently not setup to be able to add PR’s to the Victory Plugin, but I am happy to simply paste the source if that will do for now.

You’re call.

Otherwise, you’re gonna have wait at least a few days until I can get the time to sort out Git.

7

Sure! Just pasting the source code would be of a great help!

8

I would be interested in this is as well btw :smiley: just the code though
even if it’s outdated

9

Here is a simple copy-paste from me ye olde function library:


void UGBFunctionLibrary::CaptureComponent2D_DeProject(class USceneCaptureComponent2D* Target, const FVector2D& ScreenPos, FVector& OutWorldOrigin, FVector& OutWorldDirection)
{
    if ((Target == nullptr) || (Target->TextureTarget == nullptr))
    {
        return;
    }

    const FTransform& Transform = Target->GetComponentToWorld();
    FMatrix ViewMatrix = Transform.ToInverseMatrixWithScale();
    FVector ViewLocation = Transform.GetTranslation();

    // swap axis st. x=z,y=x,z=y (unreal coord space) so that z is up
    ViewMatrix = ViewMatrix * FMatrix(
        FPlane(0,    0,    1,    0),
        FPlane(1,    0,    0,    0),
        FPlane(0,    1,    0,    0),
        FPlane(0,    0,    0,    1));

    const float FOV = Target->FOVAngle * (float)PI / 360.0f;

    FIntPoint CaptureSize(Target->TextureTarget->GetSurfaceWidth(), Target->TextureTarget->GetSurfaceHeight());

    float XAxisMultiplier;
    float YAxisMultiplier;

    if (CaptureSize.X > CaptureSize.Y)
    {
        // if the viewport is wider than it is tall
        XAxisMultiplier = 1.0f;
        YAxisMultiplier = CaptureSize.X / (float)CaptureSize.Y;
    }
    else
    {
        // if the viewport is taller than it is wide
        XAxisMultiplier = CaptureSize.Y / (float)CaptureSize.X;
        YAxisMultiplier = 1.0f;
    }

    FMatrix    ProjectionMatrix = FReversedZPerspectiveMatrix (
        FOV,
        FOV,
        XAxisMultiplier,
        YAxisMultiplier,
        GNearClippingPlane,
        GNearClippingPlane
        );

    const FMatrix InverseViewMatrix = ViewMatrix.InverseFast();
    const FMatrix InvProjectionMatrix = ProjectionMatrix.Inverse();

    const FIntRect ViewRect = FIntRect(0, 0, CaptureSize.X, CaptureSize.Y);

    FSceneView::DeprojectScreenToWorld(ScreenPos, ViewRect, InverseViewMatrix, InvProjectionMatrix, OutWorldOrigin, OutWorldDirection);
}

void UGBFunctionLibrary::Capture2D_DeProject(class ASceneCapture2D* Target, const FVector2D& ScreenPos, FVector& OutWorldOrigin, FVector& OutWorldDirection)
{
    if (Target)
    {
        CaptureComponent2D_DeProject(Target->GetCaptureComponent2D(), ScreenPos, OutWorldOrigin, OutWorldDirection);
    }
}

I can’t take any real credit for the guts of the function, which I think is taken from another area of the UE4 source and simply adjusted to suite use the scene components info.

2 Likes
10

Hey @Kris, thanks a LOT for the code! Now combining both Project and Deproject functions I finally created my minimap the way I wanted it to be. It’s a shame these functions are not built-in in the engine by default.

11

Glad it worked! Enjoy!