Hi,
I have a SceneCapture2D actor capturing my scene, and the output is displayed in a widget. When the user click on the widget, I would like to know where she clicked in the 3D space.
Basically “Deproject Screen to World” in blueprint, but for a SceneCapture2D.
I found an old post about it, but the hack doesn’t work.
Any idea about how I could do that ?
Hey did you find a solution? I only found a solution if the camera has the same field of view and ratio than the player camera. I have just calculated the relative click location from the player camera and than added the SceneCapture2D transforms to it. It works but only when both cameras have the same settings… Could you find a general solution…?
Maybe this will help:
https://answers.unrealengine.com/questions/528282/linetracebychannel-and-gethitresultundercursorbych.html
I donated Blueprint functions to Rama’s Victory plugin that project/deproject second capture component 2d’s.
Provided they still work, its probably your best place to start.
Hey Kris, you said that you contributed SceneCaptureComponent2D Project/Deproject functions, but I only see the Project function. Right now I desperately need a Deproject function to be able to Line Trace from my SceneCaptureComponent2D when I click in a Widget which renders a RenderTarget texture. There are so many requests in AnswerHub but no solutions. Can you help me (and others) with that one?
Took a quick look on GitHub at Rama’s Victory Plugin and yeah, couldn’t see the Deproject functions.
Odd, coz, well, I swear I included them when I included Project 
I am currently not setup to be able to add PR’s to the Victory Plugin, but I am happy to simply paste the source if that will do for now.
You’re call.
Otherwise, you’re gonna have wait at least a few days until I can get the time to sort out Git.
Sure! Just pasting the source code would be of a great help!
I would be interested in this is as well btw 
just the code though
even if it’s outdated
Here is a simple copy-paste from me ye olde function library:
void UGBFunctionLibrary::CaptureComponent2D_DeProject(class USceneCaptureComponent2D* Target, const FVector2D& ScreenPos, FVector& OutWorldOrigin, FVector& OutWorldDirection)
{
if ((Target == nullptr) || (Target->TextureTarget == nullptr))
{
return;
}
const FTransform& Transform = Target->GetComponentToWorld();
FMatrix ViewMatrix = Transform.ToInverseMatrixWithScale();
FVector ViewLocation = Transform.GetTranslation();
// swap axis st. x=z,y=x,z=y (unreal coord space) so that z is up
ViewMatrix = ViewMatrix * FMatrix(
FPlane(0, 0, 1, 0),
FPlane(1, 0, 0, 0),
FPlane(0, 1, 0, 0),
FPlane(0, 0, 0, 1));
const float FOV = Target->FOVAngle * (float)PI / 360.0f;
FIntPoint CaptureSize(Target->TextureTarget->GetSurfaceWidth(), Target->TextureTarget->GetSurfaceHeight());
float XAxisMultiplier;
float YAxisMultiplier;
if (CaptureSize.X > CaptureSize.Y)
{
// if the viewport is wider than it is tall
XAxisMultiplier = 1.0f;
YAxisMultiplier = CaptureSize.X / (float)CaptureSize.Y;
}
else
{
// if the viewport is taller than it is wide
XAxisMultiplier = CaptureSize.Y / (float)CaptureSize.X;
YAxisMultiplier = 1.0f;
}
FMatrix ProjectionMatrix = FReversedZPerspectiveMatrix (
FOV,
FOV,
XAxisMultiplier,
YAxisMultiplier,
GNearClippingPlane,
GNearClippingPlane
);
const FMatrix InverseViewMatrix = ViewMatrix.InverseFast();
const FMatrix InvProjectionMatrix = ProjectionMatrix.Inverse();
const FIntRect ViewRect = FIntRect(0, 0, CaptureSize.X, CaptureSize.Y);
FSceneView::DeprojectScreenToWorld(ScreenPos, ViewRect, InverseViewMatrix, InvProjectionMatrix, OutWorldOrigin, OutWorldDirection);
}
void UGBFunctionLibrary::Capture2D_DeProject(class ASceneCapture2D* Target, const FVector2D& ScreenPos, FVector& OutWorldOrigin, FVector& OutWorldDirection)
{
if (Target)
{
CaptureComponent2D_DeProject(Target->GetCaptureComponent2D(), ScreenPos, OutWorldOrigin, OutWorldDirection);
}
}
I can’t take any real credit for the guts of the function, which I think is taken from another area of the UE4 source and simply adjusted to suite use the scene components info.
Hey @Kris, thanks a LOT for the code! Now combining both Project and Deproject functions I finally created my minimap the way I wanted it to be. It’s a shame these functions are not built-in in the engine by default.
Glad it worked! Enjoy!
Good thread. Has this been turned into a node now because I can see one called ‘Deproject screen to capture world’
Only issue is I have no idea what to put into the ‘scene capture 2d’ pin and the ‘target uv’ pin lol
Hey @Kris sorry to necro this but I tried using this in 5.4 with a scene capture 2d actor placed above the world. I have a widget that is using a material to display the render target from the scene capture 2d actor. I keep getting inaccurate results.
Here is what I’m doing:
My line traces are always in the wrong area and they are not even close to the scale of the viewable space in the render target from the scene capture actor. However they do create the shape of the clicked widget when clicking the mouse over different parts of the widget. For example, if I click at each corner of the widget I can tell that all 4 line traces are spaced out to the correct proportion that the widget is, just that they are super small in the world and not in the correct locations.
Is there something I’m missing here?
I don’t know enough about this kind of math to know if there is something I’m doing wrong.
Also I have noticed that if I normalize the mouse click from the widget size and then map the x and y those normalized values from (0 to 1) to (-1 to 1) and multiply that by the render target size and pass that into the screen position parameter I get way more accurate line traces. However doing that causes the line traces to all be offset to the left from where I feel like I’m clicking based on the visual from the render target in the widget.
Code was already old when I posted it back then.
Haven’t touched or used it since, so I have no clue now 