Convert 3D hit location to 2D character image

Hello,
I want to create something like this on the image. I have 3D world space hit location and I’d like to show that location on the character image in the viewport, visually show last hit location on exact spot on the character. What is the right way of doing it, hence I tried converting 3D location to meshspace and then in the widget I tried to project that to the Screen Space (also tried and Covert to ScreenSpace) and it didn’t work out.

The only way I could I think of would be to detect which bone it got hit at, and if you wanna get more detailed, place more collisions around the body.

Okay so my answer above is what I am able to achieve myself, without further research. And I can tell you the steps in more detail if you struggle on that method. But I just got another idea the second after I posted this, though don’t know how exactly to do it. So you can also place a rectangle behind your 3D character and have it rotate along with it, have your line trace (you’re probably using line trace for shooting, right?) go through your model and hit that instead. Then you can easily have the hit location show up on your 2D widget, but you must first watch a tutorial on painting canvases in Unreal, because I believe that would teach you how to detect any point from a 2D surface. Hope this helps :smiling_face_with_three_hearts:

I wanned to add another note, I was searching how to do this until I realized your models can move their arms and legs, but your widget is in the idle pose. I think the exact thing you wanna achieve will take so much effort, if it’s even possible with that mesh space method you mentioned in the first place. I believe it can be done in someway, maybe by handling different parts of the body seperately, though what I understood is that you don’t just want to make that hit location conversion into a 2D space, but want that to show up where the hit would be located if the character was in idle pose on top of that

Hi, thanks for the sharing ideas. I tried with the bone itself but it didn’t work out, I still didn’t make it show on the right location on the texture - it will return 0,0 all the time. The box collision won’t work since my trace is dependant on phys material.
I don’t need an idle pose for the character, I rlly just need an image above, to have a character image in the A pose and display a hit location on it - basically copy and paste that image would be awesome :smile:

You are both correct :slight_smile:

  • The Inverse Transform node will give you the location of the hit in local space.
  • Next we convert the 3D vector to 2D - the character up is Z we need that for Y and the character left-right is Y - this is X in our widget.
  • Next we scale. The default character is 180 units high. We will have to know how high is the picture of the character in the UI to set the proper scale. Also we multiply by -1 to reverse the axis as our would Z increase up but out Y in the UI increases as you go down.
  • The widget origin is it’s up left corner. The character’s origin is in their center. “Center on widget” variable is the place on the widget where the center of the character would be.

Now the transform in this example the transformtaken from the hit actor but as @VisAgilis pointed out, the character moves their arms and legs. If you want those hits to be correct you have to take the hit bone transform instead. In that case however, you will have to know where that bone’s origin is in your widget. Bones would be weirdly rotated on the widget itself so so you’ll have to take that into account. (Unlike the actor their axis will be skewed)

Thanks for showing the code. I tried to make it like that but there’s an offset every time, and it’s not showing a correct location - using the Bone Transform, for headshot I’m getting almost a middle of the screen. Currently what I can think of, on top of my head is basically mapping all the bones in the map and their widget location :smile:
image

Yep. If you go with the bones You’ll have to know where exactly each bone is in your widget and where are its coordinate axis pointing at.

Edit: I can think of something else but it’s still a pain in the X:

  • First you store all your original bone transforms somewhere (original meaning the ones you use in the pose used for the widget). They are constant so it shouldn’t be a problem, just a chor. You can even hardcode them manually, bone by bone.
  • When hit you transform the hit location from the current bone transform to the original bone transform (I think that happens when you subtract the transforms).
  • Then you transform the hit from the bone coordinates to the character coordinates.
  • Then you are left with the issues explained above - transfer from character to widget coordinates.

Yea, map all of those and then take a hit bone origin hit origin on the character mesh, and apply offset to the widget bone position. I think that’d be the best option for now :confused:
Thanks for brainstorming this ideas :slight_smile: