How to know a point is in rectangle in XY plane?
I think UE4 may already have some function for this, but I don’t know where it is.
If I have several points in XY plane(or 4 points as rectangle), I want to know a point is in that region or rectangle.
(But, not AABB. I tried FBox and I found it’s AABB.)
I will assume that you need the math for this. (based on the question tags)
You can go in different ways but my preferred method by far is:
 Create a transform to a local coordinate system aligned with your rectangle;
 Transform your point (that you need to check) to that local coordinate system;
 Use the AABB test.
Lets say your rectangle corners are stored in FVector A, B, C, D;
and your point in FVector P;
Now before going into solving this you must first…

Ensure that your all rectangle points and the point you need to check lay in the same plane:
/* It will be much easier if you first add all your points in an array*/ TArray<FVector> Pnts = {A, B, C, D, P};
If you need them to be right on XY plane just check if Pnts[i].z == 0.0f;
If you need them to be on a plane paralel to XY just check if Pnts[i].z == RecPnts[i+1].z;
If you have them in any orientation use FMath::PointsAreCoplanar(Pnts);
 Set Origin and find vectors for the new coordinate system:
The thing you have to look out on this step is not to take the diagonal as the X axis.
/* It would be easier if we know the order of the points A,B, C and D but I will assume we don't*/
FVector Origin = A;
FVector XAxis = DotProduct( CrossProduct(B  A, C  A), CrossProduct(B  A, D  A) ) > 0.0f ? B  A : C  A;
FVector YAxis = D  A;
FVector ZAxis = FVector::CrossProduct(XAxis,YAxis);

Create a transform to a local coordinate system aligned with your rectangle and transform your point;
FTransform ToRec = FTransform(XAxis, YAxis, ZAxis, Origin);
FVector TransPnt = ToRec.TransformVector(P); 
Conduct an AABB test:
We will write the AABB test ourselves. Since we know that if we transform our rectangle with ToRect we will have the Origin A at (0,0,0) XAxis at (1,0,0) and the YAxis at (?,1,0) ("?" is because we don’t know if we’ve taken the diagonal or the other side for YAxis)
return (TransPnt.x >= 0.0f && TransPnt.x <= 1.0f) && (TransPnt.y >= 0.0f && TransPnt.y <= 1.0f);
IMPORTANT NOTE:
FTransform::FTransform( const FVector & InX, const FVector & InY, const FVector & InZ, const FVector & InTranslation )
is spectacularly badly documented. There is no information neither on what input checks are internally made, what process is used for creation nor what the output is so I assume that the resulting transformation is corrected to not produce shear and the following holds true after the transformation:
InX = (1,0,0)
InY = (?,1,0)
InZ = (?,?,1)
InTranslation = (0,0,0)
I also assume that the shape defined by A, B, C and D is at least close to a rectangle.
I hope this helps
That’s awesome. You’re my hero.
Thanks, but keep in mind I’ve not tested the code.
It is probably full of errors and there is a slight chance that FTransform does not work as I assumed.
Edit: Also you might need to define some kind of float tolerance and use that instead of == if the complanar check fails too often. You can also skip it altogether and it should work loosely.
Did it work?