Please click / download the full image link at the bottom to see this clearly.
You want to create the input direction in world space first. You do this step most likley already when you move your character around relative to the camera’s rotation (yaw only)…
Afterwards I normalize that input direction so it has a vector length or size of 1. (However if you have normalized it but the size is 0, it means you are not giving any input).
If the size is not greater than 0, it means there is no direction input. Perform “Neutral Attack”.
If the size is greater than 0, we need to Dot Product our input direction and our character forward direction vectors.
If you’re new to dot product of two vectors then all you need to know is, it goes from (float) -1 where directions are opposite (like your input is backward from character forward), 0 where they are perpendicular, and 1 where the directions are the same (like your input is forward with the character). (also assuming both vectors are normalized to 1)
Example IE you press kinda foward but not perfectly you may get a value like 0.78 or kinda backward -0.13.
So finaly if DotProduct( player direction, input direction) >= 0 do a foward attack. Else we are backward so backward attack. Just note I used >= because the dot product could be 0 or perpedicular so we still want to do something.
Really just know how to use Dot Product to compare vectors and you’ll do great things!