Motion controls to puppet hand IK bones?

I’ve recently started experimenting with the Vive motion controls and I assume these can be setup to control the IK hand bones of the player’s character. Has anyone worked with this before? How’s it worked out in your testing? I think our initial implementation will be to just hide the character mesh and have meshes of hands attached to the motion controls when they are being used but ideally, I’d like to use the IK bones to have arm motions that are tied directly to the player’s hand movements.

Not sure how this would work from a technical / implementation perspective in UE4, but here’s a cool proof-of-concept video showing IK arm motions linked with the Vive motion controllers:

yes it’s possible.
if I remember right, there’s even some example here on the forums (using a Razr Hydra, because nobody gets Vive devkits…).

I made some progress with this and we’ve just started integrating Ikinema into the project to help improve on it but there’s some limitations with the Vive software that need to be addressed before we can add this into our game. I started a new post that details two of these issues. My fingers are crossed that fixes can make it into 4.11. With our release on Steam in line with Vive and Oculus release in April, I suspect we will likely launch with 4.11.

On an unrelated note, I drove around our in-game neighborhood using the Vive for the first time last night and it was quite the experience. I even felt a bit nervous when I saw a cop pull up at the stop light :slight_smile:

Finally making some progress with this. I hit a major roadblock because the location and rotation values I was pulling from the motion controllers were not matching up with the values I needed in my IK transforms but I finally got it worked out.

For the location, Z was an easy one to determine the offset. With the motion controller components in a scene component placed at the head, I just had to figure out the distance from the scene component to the floor and add that to the Z. X and Y stumped me for a better part of the night but I finally solved the puzzle. The offset ended up being X=Y and Y(-1)=X.

Of course, it would be too easy if the same formula worked for the rotator, lol. This one required a negative offset for X/Y on the right controller with 90 added to Z and the left control had no change to X/Y with a -90 added to Z.

I already shutdown the project but I’ll grab some screenshots of this info if anyone needs it. Next step, finger controls.

As for me, the real problem now is shoulders tracking. I mean, we need separate camera and controller rotators with controller rotator following to the player’s body (and soulders!). I can’t make it work smoothly yet after whole day spent for this :frowning:

Heya I had a question for you, were the changes/ calculations you made done in Blueprints? And If so I would I be able to look at some of your screen shots to better understand how that was setup?
Best regards.

Yup, everything was done in the character BP and the animation BP.

Here’s the right hand stuff from the character BP. For testing purposes, I am only using IK when trigger is pressed so you can ignore that part. The value of 155 I’m adding to Z in the transform is because that’s the distance from the floor to the scene component at the head where the motion controls are under. This is b/c Vive seated mode takes 0,0,0 from camera location for HMD but still takes floor real-life floor reference for motion control.


In the animation BP I am getting the alphas and transforms via cast and setting them to the animationBP variables seen in the screenshot.

IK head and hands are working nicely. Now I’m working on experimental touch interaction for the in game tablet.

Hey could you please explain how you got the number 155 you added into transform location z. If that’s relative to the floor wouldn’t that change based on the persons height?

I just started using Ikinema and steamvr myself :slight_smile: any help would be awesome.

Oh i just tried with my rig and realized what you where doing there. awesome. ignore me :wink:

Hi guys, just for information, we are adding direct support to Vive in the upcoming release with a dedicated rig editor for VR …

Awesome ! i have no idea about how to set the hand’s transform as vive controller , can u give me some advice? thx !

This does look amazing ! I do have a question:

Does it support easy retargetting IK end effectors/re-calibrating for height/limb length etc?

We need the IK to work for anyone from 8 year olds to 6 foot 10 people via calibration from a click of a button without any rigid pose requirements.

The re-scaling has to be done outside of features available in the IKinema nodes and should be implemented in your game logic. This can be achieved a number of ways, one of which is described in item 6.2 of the documentation found here (along with a demo VR project containing the logic): IKinema Runtime Documentation

This example does require the user to assume a specific pose so you can accurately calculate a scaling ratio to go from real-world transforms to game-world transforms. As I mentioned, this currently goes outside the realm of the IK solving features of IKinema, but was a method we implemented which worked well.

Ahh ic. In the midst of all of this, Any news on when the VR Solver will come out with UE4?

The latest release of the Ikinema RunTime plugin includes VR solving features for UE4.12.

Ahhh alrighty. Thanks ! :slight_smile: Does this also work with custom engines? (Like Multi-Res)

EDIT I’ve tried from source, and launcher. for some reason it still doesn’t work with 4.12.5 It tells me this

0443e3345fb2df498edbfe8f7063e1d6da90bce4.jpeg

Basically is telling me it cannot find the “IKinema-Core.dll” I put everything in the right places(according to the documentation), but still not working.

The log file that produced is this:



[2016.08.08-17.24.23:708]  0]LogModuleManager:Warning: ModuleManager: Unable to load module 'IKinemaCore' because the file 'F:/UnrealEngine-4.12.5_NVIDIA_Techs/Engine/Binaries/Win64/' was not found.
[2016.08.08-17.24.23:723]  0]LogModuleManager:Warning: No filename provided for module IKinemaRigTool
[2016.08.08-17.24.23:724]  0]LogModuleManager:Warning: ModuleManager: Unable to load module 'IKinemaRigTool' because the file 'F:/UnrealEngine-4.12.5_NVIDIA_Techs/Engine/Binaries/Win64/' was not found.


I’ve also tried this on 4.11 and same thing happens… Really curious what I’m missing here :frowning:

The indie plugin does not support source builds of unreal or custom engines, so the errors in the log you posted are expected. However, it should work fine if you are using it with the binary build of unreal engine. If you are OK with using the binary build, could you give that a go and report back.