Making Realistic Player Interaction in VR (Tutorial)

Hello UE4 Community.
I have been working on making realistic sword fighting mechanics for VR in Unreal Engine 4 for a while now.

My goal was to make a system that can be replicated and used by others, and also improved upon. Because of that I did not want to use plugins and opted to use blueprints instead.

It has been many months since I have been working on this project, I am a novice in unreal engine and have reached the end of what I can do, and have reached the end of what I can understand through the use of documentation, tutorials, and trial and error.

I am going to post all my blueprints and work, and my hope is that you guys can improve upon it, learn from it, discuss about it, and ultimately work on it together or even individually and take it further i could have ever imagined. My only request is that if you do improve it please share your work with us so that everyone benefits, and we become closer to making VR sword fighting realistic.

Remember all these quotes by great people:
“Alone we can do so little; together we can do so much.” – Helen Keller

“None of us, including me, ever do great things. But we can all do small things, with great love, and together we can do something wonderful.” – Mother Teresa

“Many ideas grow better when transplanted into another mind than the one where they sprang up.” – Oliver Wendell Holmes

“If I have seen further, it is by standing on the shoulders of giants.” – Isaac Newton

Things to read before hand:
The static mesh that I was using for the sword was made by me, I tried to make it as accurate to a real sword as I could in terms of basic dimensions.
I’ll post it here:
https://drive.google.com/drive/folders/1IKnT8MFuJ9mA2uBP1S6zFZNTQW-_0xfC?usp=sharing

I have also posted the project file there for those who want to play with it and learn from it:

There are essentially two ways that you can start with, the first is that you could attach the weapon to a socket or mesh and go from there. The second way is by attaching the weapon to a physics handle.
Both have there pro’s and con’s

Attach to mesh or socket:
Pros:

  • Easy to do.
  • Sword does not lag behind
  • Initially less complicated

Cons:

  • Sword goes through non-blueprint objects
  • Will have to make some complicated system of how the sword should react when it hits another sword, or a wall.

Attach weapon to Physics Handle
Pros:

  • Matthew palaje has a good tutorial on how to do this
  • You can try to use the already existing physics system for the sword, maybe after some tweaks you can get it to work
  • Plenty of documentation on the physics system

Cons:

  • I don’t have a very good idea of how the physics system works

I began with attaching the weapon to a socket as it seemed to be the simpler route.
This is the blueprint I made for this type of sword.
HeroWeapon_BP - The weapon is attached to the socket or mesh (you can attach it to either one).


In order to release the weapon you will have to add a few things to the “Release Actor” function in BP_MotionController
[ATTACH=JSON]{“data-align”:“none”,“data-size”:“full”,“data-tempid”:“temp_155327_1547095950321_567”,“title”:“2.PNG”}[/ATTACH]

But after this step it seemed impossible to fix a few of the problems that occured:

  • The sword kept going through objects and walls
  • The sword did not have any realistic interaction with the objects around it
  • I did not see how i could create a physics system for this

**So I decided to attach the weapon to the physics handle instead: **

To understand what I did I suggest you watch this tutorial, my work was similar to Matthew Palaje’s:
You basically have to watch this video in order to understand what is going on, and to be able to follow what I did.

BP_WeaponMaster

  • Is the Physics Master as seen in the video

  • My blueprints are slightly different than that of the video as we have two slightly different goals, but the concept is nevertheless the same. Let me give you an explanation of what I did.

  • This blueprint will be the master blueprint, from this I will make child classes who will inherit the blueprint and properties from this. You can then go and tweak each child individually as you like. Whatever happens to the master will happen to the child. This is good if you wanted to have multiple weapons all of which use the same things.

  • You want to begin by making a custom collision channel, so go into Edit > Project Settings, under Engine go to Collision and make new object channel, call it weapon and set the default response to block. That way the sword by default will not go through anything. And later on you can set custom collision responses for the sword.

  • While you are at it you will want to also make an object channel called “VRHand”, and set the default response to block. This is important as we will need to give the hand a custom response so that it does not collide with the sword when it is grabbing the sword.
    [Picture of Object Channel]

  • Go BP_MotionController and in “HandMesh” change the setting so that the object type is VRHand and everything is blocked by default.

  • We do not need to use line traces or check for whether the object is valid or not as that is done for us in the “Get Actor near Hand” function in the BP_MotionController class. And the “Grab Actor” function will grab the weapon for us.
  • Going back to the BP_WeaponMaster:
    The weapon is attached to the physics handle. When the player presses the grab button on his controller the physics handle teleports to where the hand is. Then the sword teleports and stays attached to the physics handle. We created a loop so that as long as the hand is grabbing, the physics handle follows wherever the hand goes.
  • Obviously we need a way to drop the weapon. So on “event drop” the physics handle is released, isGrabbing is set to false, the weapon is allowed to collide with the hand once again and the loop is told to stop.
  • You will also want to change the Collision settings of the weapon so that the object type is “Weapon”. Set the collision with the “VRHand” to “Ignore” as we don’t want to keep detecting or colliding with it.
  • We will also need to make the change in the “Release Actor” function in BP_MotionController, so that the hand stops holding the weapon.
  • You will notice that the sword acts kind of weird and slow, in order to make that more realistic you can set the “Soft Angular Constraint” and the ”Soft linear Constraint” under “Physics Handle” to off.
    Object Acting Slow After Attaching it to Physics Handle - YouTube

	https://youtube.com/watch?v=yULXOjrqdJM
  • As you can see from the previous video even this mechanic is not perfect. When the sword collides with the walls it bugs out as it does not know where to go or what to do.You can improve this bug by going into the PhysicsHandle of the BP_WeaponMaster class and enabling the soft angular constraint. By doing this the sword will rotate in your hand when it collides with objects. Go ahead and try it.
    Making Object More Rotating After Attaching it to Physics Handle - YouTube

  • Even so it still acts weird, this can be further improved by Decreasing Angular Damping to a relatively smaller number and increasing Angular Stiffness to some monstrously huge number. I did 100 for the first and 1,000,000 for the second.

  • I got that result by just playing around with the numbers, I encourage you all to play with them and understand them further to get better results.

  • Here is what that improvement looks like.
    Making Object Act More Realistic After Attaching it to Physics Handle - YouTube

  • If you want you can make a child blueprint class from this master class by right-clicking on the BP_WeaponMaster and clicking “Create Child Blueprint Class”, and it will act exactly the same way as the master. But you can change it and tweak it without it affecting the master
    BP_SwordWeapon - inherits its blueprints from weapon master

Congratulation you now have a semi realistic swordplay mechanic. Click play and try it out.
Remember if you find ways to improve this please share your finding so that we can get one step closer to making sword fighting and interactions more realistic.

and most importantly: Whenever you make a change and try out your new sword or object make sure to throw it as far as you can. JK, but it is fun to do.

You might want to try either modifying the center of mass (COM) of the sword to be at the grip point or dynamically setting it (modifying the COM offset to move it there) when gripping to that location. You get a lot more stable simulation and a correct pivot off of doing that. When the COM is offset from the grip the physics handle is fighting the weight from that offset at the same time as it is trying to move the body.

Also your videos are broken.

Thanks for the feedback, I have hopefully fixed the broken links and will try to change the center of mass and try again. Thanks

How do I dynamically change COM?

My problem is no matter how I set the center of mass offset off the hand, it still translates (not just rotates around its center) as I apply Angular Stiffness = 30,000 It looks like as if it is doing orbital motion around the controller when the user slightly rotates the controller.