Plugin

[getnamo]

Thought I’d share how I have the body leaning setup as used in the Gifs I shared earlier.

Example Result

How it was done:

First Step, getting the HMD, placed inside Character (Pawn). Fetch the values every tick and store them into local variables Head Rotation and Head Offset. Ignore Head velocity that is an experimental value used for determining jumping (kinda works :rolleyes:). Yaw offset is only used to account for misaligned camera/hmd.

Second Step, inside the animation blueprint Event Graph, grab the values found in the pawn rotate them to align with the local axis (they use Y as forward for some reason).

Third and Final Step, inside the animation blueprint Animation Graph, use the values with a combination of rotations and IK to adjust shoulder roll, leaning, and head rotation. The graph is pretty messy right now as it was used for a more of a proof of concept than a production environment, but it works. Note that the head rotation is split over two bones (neck and head at 30/70 split) to provide a more realistic head roll. The body leaning is not an ideal setup, but the values used here give a natural leaning by scaling the headoffset and forwarding that both as a Two Bone IK and a rotation for the spine_03 bone. Reach Rotation is used to add additional yaw rotation based on player trying to reach past current pose with their arms see this example use case.

It would be interesting to see how the setup can be improved upon as I’m using a lot of ‘feel good’ variables and I’m sure there is a more consistent approach.

[n00854180t]

[=;160086]
It will work but remember that you need to launch standalone and you require the app to go fullscreen before it reports the position, otherwise as far as UE4 is concerned the HMD is off.

[/]

Was already doing that. No luck… HMD position is always 0, 0, 0, even when it obviously has tracking (I’m just printing the value, since I can’t debug in Blueprint due to the Standalone window thing).

Anyway, I’ll try your method for the body leaning, though I suspect it will act wrong due to HMD position always being zero.

Edit: Got my project moved to 4.5 preview (from 4.3.1 with 0.4.2 SDK support - the Oculus branch), so hopefully that will get me the position data.

[anonymous_user_68a6361c]

[=;159797]
THe of it is to just replace the two bone IK nodes with a fabric node, and do a Make Transform (plug in the position and rotation of the hydra per hand). Set the chain tip to be the hand, the root to be the upper arm.

Then set the rotation to “copy target rotation” - that makes it so you don’t need the “Transform (Modify) Bone” nodes.

Also multiply the position of the hydra by 0.5, no idea why that works but it does (may or may not work with the regular setup).
[/]

Okay so that good news is I was able to set up both methods inside the animation blueprint. The bad news is I must have missed a step somewhere along the way because neither method has worked yet. I’m moving the hydras around like a crazy person and the hands continue to not move. From what I’ve been able to debug, I think the variables I set up for the hand based on the hydra position inside the Animation Blueprint are not being read correctly for some reason (probably through some fault of my own that I haven’t figured out yet). All of my graphs for the Hydra Player Controller are exactly like the ones you showed me as is the Event graph in the Animation Blueprint with the exception of a Cast to MyCharacterExplorationNoHeadBob node. I’ve noticed at least that when I manually tweak the variables in the animation blueprint editor, the positions of the hands will change (finally!) after setting up the bones using the FABRIK nodes but I can’t get that to happen in game using the hydra. I know is somewhat confusing, but if you have any idea what I could be doing wrong, could you lend me a hand? I have the ability to post screenshots too but I didn’t want to spam the with them just yet.

[n00854180t]

[=aialexander;160381]
Okay so that good news is I was able to set up both methods inside the animation blueprint. The bad news is I must have missed a step somewhere along the way because neither method has worked yet. I’m moving the hydras around like a crazy person and the hands continue to not move. From what I’ve been able to debug, I think the variables I set up for the hand based on the hydra position inside the Animation Blueprint are not being read correctly for some reason (probably through some fault of my own that I haven’t figured out yet). All of my graphs for the Hydra Player Controller are exactly like the ones you showed me as is the Event graph in the Animation Blueprint with the exception of a Cast to MyCharacterExplorationNoHeadBob node. I’ve noticed at least that when I manually tweak the variables in the animation blueprint editor, the positions of the hands will change (finally!) after setting up the bones using the FABRIK nodes but I can’t get that to happen in game using the hydra. I know is somewhat confusing, but if you have any idea what I could be doing wrong, could you lend me a hand? I have the ability to post screenshots too but I didn’t want to spam the with them just yet.
[/]

Are you doing the calibrate step correctly? Stand in a t-pose (arms extended to the sides, shoulder height) and press the start button (or whatever button you used in your controller that the calibrate function is called from).

[n00854180t]

[=;160093]

Second Step, inside the animation blueprint Event Graph, grab the values found in the pawn rotate them to align with the local axis (they use Y as forward for some reason).

[/]

What’s the definition for Convert Rotation? Thanks man, almost got set up.

[getnamo]

[=;160444]
What’s the definition for Convert Rotation? Thanks man, almost got set up.
[/]

Also note that character owner has to be obtained and cast to your pawn type so that you can access the head/hand variables. I store pointer as character owner so I can reference it later.

[anonymous_user_68a6361c]

[=;160430]
Are you doing the calibrate step correctly? Stand in a t-pose (arms extended to the sides, shoulder height) and press the start button (or whatever button you used in your controller that the calibrate function is called from).
[/]

I have tried this. The result is still the same. I will double check my graphs again later today and see if maybe I can spot something with fresh eyes.

EDIT: I think the errors I am getting have to do with the fact that I am trying to build on someone else’s already established code in the case of the Rift Template. I’ve noticed that when I tried to switch the VR controller with the HydraPlayerController in the world settings, I began to encounter issues, such as errors coming up from all of the references to the VR controller in the character Blueprint. It may take me recreating all of those VR Controller variables in the Hydra Controller just to get those errors to subside. That is my theory anyway.

[n00854180t]

[=aialexander;160758]
I have tried this. The result is still the same. I will double check my graphs again later today and see if maybe I can spot something with fresh eyes.

EDIT: I think the errors I am getting have to do with the fact that I am trying to build on someone else’s already established code in the case of the Rift Template. I’ve noticed that when I tried to switch the VR controller with the HydraPlayerController in the world settings, I began to encounter issues, such as errors coming up from all of the references to the VR controller in the character Blueprint. It may take me recreating all of those VR Controller variables in the Hydra Controller just to get those errors to subside. That is my theory anyway.
[/]

Did you just switch from one to the other?

You should probably Reparent one of them. There are multiple ways to set it up.

If you have a controller that already is a child of HydraPlayerController, you could make VRPlayerController a child of that.

Otherwise you could just make VRPlayerController a child of HydraPlayerController, then reimplement any of your Hydra related code. I suggest that way myself.

Here, here’s some screenshots explaining.

Open VRPlayerController.

Select File -> Reparent Blueprint.

Then select either HydraPlayerController (and subsequently implement the Hydra-related stuff into VRPlayerController, which will have access to the Hydra BP nodes), or select your already made PlayerController that implements the Hydra code.

[getnamo]

You can also use the HydraPluginActor, place it in the scene and get a pointer to the player controller in event begin play and use that reference inside the plugin actor. From there you forward your hydra input into variables in the player controller.

was one of the reasons why I made a convenience actor, allowing you to extend functionality without changing any of the already implemented code. It is also how you can have control indirection, since the variables such as Left/Right Hand Position inside your player controller can be updated by different devices via plugins similar to the Hydra Plugin and the player controller doesn’t care how the positions were obtained.

[anonymous_user_68a6361c]

[=;160954]
Did you just switch from one to the other?

You should probably Reparent one of them. There are multiple ways to set it up.

If you have a controller that already is a child of HydraPlayerController, you could make VRPlayerController a child of that.

Otherwise you could just make VRPlayerController a child of HydraPlayerController, then reimplement any of your Hydra related code. I suggest that way myself.

Here, here’s some screenshots explaining.

Open VRPlayerController.

Select File -> Reparent Blueprint.

Then select either HydraPlayerController (and subsequently implement the Hydra-related stuff into VRPlayerController, which will have access to the Hydra BP nodes), or select your already made PlayerController that implements the Hydra code.

[/]

I gave a shot by parenting the VR Controller to the HydraPlayerController and combining the code for the hydra that I had already programmed in another blueprint by recreating it in the VR Controller Blueprint and I managed to get the errors to stop. However, I still have not been able to get the character’s arms to respond to the Hydra. I will keep trying and let you all know if I manage to come up with anything. If you happen to get that tutorial ready, I could really use it. had a suggestion for me as well, so I’m going to look over his instructions again to see if I can understand it a little better.

[n00854180t]

I almost have the body setup working, but can’t seem to get it so that if you lean back with the Rift, it will lean the torso correctly enough to prevent clipping through the head. I could just hide the head, but there would still be issues… I’d like to get it such that the head follows properly enough that you just don’t see it (and it almost works well enough to do that now).

@ - How are you doing the Reach Rotation Yaw and Reach Alpha values? I assume you’d want to check if the previous IK didn’t meet the target, but I don’t see an obvious way of doing that from the IK nodes. Or are one or both just statically set?

Edit: I think I’ve improved the basic setup a little bit by using FABRIK instead of the Two-Bone IK for the “spine3” IK. I still have some problems right now, such as the HMD basically being rotated 90 degrees in terms of view compared to the player, but that should be fixable with the offset (which I haven’t messed with). I have some other problems in my setup I need to fix.

Unfortunately due to how long is taking to really get right, it’s going to take me a bit longer to get the tutorial up.

[getnamo]

@ The reach is done the following way:

Again using a bunch of feel good variables, but it works

[]
I almost have the body setup working, but can’t seem to get it so that if you lean back with the Rift, it will lean the torso correctly enough to prevent clipping through the head. I could just hide the head, but there would still be issues… I’d like to get it such that the head follows properly enough that you just don’t see it (and it almost works well enough to do that now).
[/]

You will always have to compromise on a little if you use any form of animation. The third person character adds a head which 90% of the users don’t want, but it makes for better external animation (I’ve tried attaching the camera to the head, its usable, but only for acclimated developers in vr, and generally prefer it without). For any high head motions where the user’s real head doesn’t you will need to have the head slightly desynced from the body, ideally we need a simple way to mask out the head. I’m thinking to maybe use a masked material with a variable that only triggers true for owner, either that or having split geometry (head/body separate, head ownersee = false). The advantage of the masked material is that can be vertex painted (requiring no external geometry modification), whereas the separate geometry is probably better for performance. I haven’t tested either so far.

[]
Edit: I think I’ve improved the basic setup a little bit by using FABRIK instead of the Two-Bone IK for the “spine3” IK. I still have some problems right now, such as the HMD basically being rotated 90 degrees in terms of view compared to the player, but that should be fixable with the offset (which I haven’t messed with). I have some other problems in my setup I need to fix.
[/]

Nice, curious to see in action! One thing to remember for the HMD is that by default it is attached to your control rotation. Using oculus separate view code you regain the ability to rotate the hmd fully arbitrarily, which is what I use as a base. I believe the VR template uses a blueprint solution with counter rotations to the mesh, which I’m not a fan of. Unsure if would impact your HMDOffset, but wanted to make sure you got your bases covered.

Also I’ve recently made an event-driven Leap Motion plugin which taught me some things as I experimented with interfaces and learned about C++ component registration. Will feed back some of that architecture back into the hydra plugin (convenience classes will remain unchanged functionally). will mean being able to add a ‘HydraComponent’ to any blueprint and after adding an interface and setting self as delegate will allow you to receive hydra events to any blueprint. would solve any re-parenting issues in the future and allow full flexibility of using input devices. would for example, allow multiple input plugins that support architecture, to be added to any blueprint and receive all of their events.

[getnamo]

Updated plugin to v0.7
-Updated to UE4.5
-New Component based system using interfaces; now able to extend any
blueprint class to have a Hydra component and receive the HydraInterface
events! See readme or unreal thread for details.
-All events now emit a UHydraSingleController, allowing for easy probe
of additional data at the event state.
-Some compatibility may be broken, but old style C++ extension is still
possible using the old delegate (still retained). New style C++ plugin
extension leads to less code, see convenience actor for details.

Component Based System
method works by adding a Hydra Component and then subscribing to the events by adding a HydraInterface to your interfaces in your blueprint class. Both parts are required for you to receive hydra events.

Setup
Open the blueprint you wish to receive hydra events.

Add the Hydra Component to your blueprint through method 1 or 2.

(Option 1) Add component directly

(Option 2) Add component through event graph

To receive events we now have to add an interface. Click on Blueprint Props and under Details find ‘Add’ under interfaces. Add the HydraInterface.

How to Use

You can get any of the HydraInterface events by right clicking on the event graph and typing ‘Hydra events’

An event will emit when the action occurs, giving all relevant data in context as pins in the node. You can filter the events by hand using Is x Hand function on the controller output.

The controller pointer is emitted for every relevant event, allowing you to conveniently access other variables you may be interested in that context. removes the need to poll for most cases.

If you need to access past frames you can do so by querying your HydraComponent. The plugin stores the last 10 frames for use i.e. gesture recognition. The controllerId refers to each single controller, 0 or 1 is a valid input. You can also manually check if your hydra is available. Should the state change it will also emit an event since the plugin supports hot plugging.

Feedback is always welcome, seeing as might break a few things, let me know if setup makes things easier or harder for you

Edit:
Update to 0.7.1
-Synced folder layout with other plugins. To install you now need to copy both Binaries and Plugins for future plugin updates.
-Added support for 32bit shipping build, for either packaged or shipping build copy only the Binaries folder after packaging has been complete. See github instructions for details

[anonymous_user_68a6361c]

Hey, ! I just wanted to let you know that I tried out your new system and went through process for the new hydra plugin, but I seem to have broken something. Now every time I try to hit play in the editor, Unreal crashes. All I did was add the Hydra component to the default character blueprint. I will try the second option and see if I get the same result.

EDIT: Okay I’ve tried both methods and I’ve gotten crashes for both. Not sure what’s causing it. For the time being I’ll see what I can do with the older tutorials for plugin and see how far I get.

[anonymous_user_68a6361c]

By the way, may be slightly off topic, but would anyone be able to tell me what a poseable mesh is or how to go about creating one in Unreal? While reading a tutorial for the leap motion, I noticed was one of the components, but I could not find any reference to what would be in any of the Unreal documentation or forums.

[getnamo]

[=aialexander;165424]
Hey, ! I just wanted to let you know that I tried out your new system and went through process for the new hydra plugin, but I seem to have broken something. Now every time I try to hit play in the editor, Unreal crashes. All I did was add the Hydra component to the default character blueprint. I will try the second option and see if I get the same result.

EDIT: Okay I’ve tried both methods and I’ve gotten crashes for both. Not sure what’s causing it. For the time being I’ll see what I can do with the older tutorials for plugin and see how far I get.
[/]

If you’re using 0.7.1 make sure you’re copying both the **Plugins **folder and the Binaries folder. The update brought win32 shipping build and I synced the overall setup with all my other plugins. I’ve tested the plugin on a blank project and it shouldn’t crash, if it still is, send me your logs, but before you do that try to start from a fresh project so you can help narrow the scope.

[anonymous_user_68a6361c]

Alrighty, I gave that a shot as well, and even started a fresh project. Still crashing. How do I send you my logs?

[getnamo]

[=aialexander;165549]
Alrighty, I gave that a shot as well, and even started a fresh project. Still crashing. How do I send you my logs?
[/]

is weird, the hydra plugin has always been very stable. To help me narrow the issue, make a clean project (no code), add the plugins and binaries then open the project. Then add the Hydra Component and add the Hydra Interface (edit: part is NOT optional for receiving events!). Then in your graph try subscribing to any of the events. Hit play in editor, if you crash then send the logs found at {Project Root}/Saved/Logs.

[MaxFlash]

I have the same problem. (crash)

For me the FhydraPlugin::DelegateTick is called before the fHydraPlugin::SetDelegate

Putting in a flag in SetDelegate and testing it in DelegateTick stops the first crash on startup.

If stopped and restarted in the editor it will again crash because the DelegateTick is call before SetDelegate.

Is there a function to test the Delegate to make sure it has a good value?

[getnamo]

[=MaxFlash;165786]
I have the same problem. (crash)

For me the FhydraPlugin::DelegateTick is called before the fHydraPlugin::SetDelegate

Putting in a flag in SetDelegate and testing it in DelegateTick stops the first crash on startup.

If stopped and restarted in the editor it will again crash because the DelegateTick is call before SetDelegate.

Is there a function to test the Delegate to make sure it has a good value?
[/]

You haven’t set the Hydra interface, if you’re using components you must add the hydra interface to your blueprint. There should be a warning shown and in the log saying “HydraBlueprintDelegate Warning: Delegate is NOT set, did your class implement HydraInterface?”. I will ensure won’t cause a crash in the future, but is the root cause of the problem.

Edit:
**Patched the plugin to 0.7.2 **which contains the fix. will stop your game from crashing, but until you add the HydraInterface to your class, you will not receive the hydra events and a warning will be emitted to your screen and log files.