Announcement

Collapse
No announcement yet.

Razer Hydra Plugin

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    Thought I'd share how I have the body leaning setup as used in the Gifs I shared earlier.

    Example Result

    How it was done:

    First Step, getting the HMD, placed inside Character (Pawn). Fetch the values every tick and store them into local variables Head Rotation and Head Offset. Ignore Head velocity that is an experimental value used for determining jumping (kinda works ). Yaw offset is only used to account for misaligned camera/hmd.



    Second Step, inside the animation blueprint Event Graph, grab the values found in the pawn rotate them to align with the local axis (they use Y as forward for some reason).



    Third and Final Step, inside the animation blueprint Animation Graph, use the values with a combination of rotations and IK to adjust shoulder roll, leaning, and head rotation. The graph is pretty messy right now as it was used for a more of a proof of concept than a production environment, but it works. Note that the head rotation is split over two bones (neck and head at 30/70 split) to provide a more realistic head roll. The body leaning is not an ideal setup, but the values used here give a natural leaning by scaling the headoffset and forwarding that both as a Two Bone IK and a rotation for the spine_03 bone. Reach Rotation is used to add additional yaw rotation based on player trying to reach past current pose with their arms see this example use case.




    It would be interesting to see how the setup can be improved upon as I'm using a lot of 'feel good' variables and I'm sure there is a more consistent approach.
    Last edited by getnamo; 10-07-2014, 08:50 AM.
    Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

    Comment


      Originally posted by getnamo View Post
      It will work but remember that you need to launch standalone and you require the app to go fullscreen before it reports the position, otherwise as far as UE4 is concerned the HMD is off.
      :P Was already doing that. No luck... HMD position is always 0, 0, 0, even when it obviously has tracking (I'm just printing the value, since I can't debug in Blueprint due to the Standalone window thing).

      Anyway, I'll try your method for the body leaning, though I suspect it will act wrong due to HMD position always being zero.

      Edit: Got my project moved to 4.5 preview (from 4.3.1 with 0.4.2 SDK support - the Oculus branch), so hopefully that will get me the position data.
      Last edited by n00854180t; 10-07-2014, 01:45 PM.
      Storyteller - An immersive VR audiobook player

      Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

      Comment


        Originally posted by n00854180t View Post
        THe TL;DR of it is to just replace the two bone IK nodes with a fabric node, and do a Make Transform (plug in the position and rotation of the hydra per hand). Set the chain tip to be the hand, the root to be the upper arm.

        Then set the rotation to "copy target rotation" - that makes it so you don't need the "Transform (Modify) Bone" nodes.

        Also multiply the position of the hydra by 0.5, no idea why that works but it does (may or may not work with the regular setup).

        Okay so that good news is I was able to set up both methods inside the animation blueprint. The bad news is I must have missed a step somewhere along the way because neither method has worked yet. I'm moving the hydras around like a crazy person and the hands continue to not move. From what I've been able to debug, I think the variables I set up for the hand based on the hydra position inside the Animation Blueprint are not being read correctly for some reason (probably through some fault of my own that I haven't figured out yet). All of my graphs for the Hydra Player Controller are exactly like the ones you showed me as is the Event graph in the Animation Blueprint with the exception of a Cast to MyCharacterExplorationNoHeadBob node. I've noticed at least that when I manually tweak the variables in the animation blueprint editor, the positions of the hands will change (finally!) after setting up the bones using the FABRIK nodes but I can't get that to happen in game using the hydra. I know this is somewhat confusing, but if you have any idea what I could be doing wrong, could you lend me a hand? I have the ability to post screenshots too but I didn't want to spam the forum with them just yet.

        Comment


          Originally posted by aialexander View Post
          Okay so that good news is I was able to set up both methods inside the animation blueprint. The bad news is I must have missed a step somewhere along the way because neither method has worked yet. I'm moving the hydras around like a crazy person and the hands continue to not move. From what I've been able to debug, I think the variables I set up for the hand based on the hydra position inside the Animation Blueprint are not being read correctly for some reason (probably through some fault of my own that I haven't figured out yet). All of my graphs for the Hydra Player Controller are exactly like the ones you showed me as is the Event graph in the Animation Blueprint with the exception of a Cast to MyCharacterExplorationNoHeadBob node. I've noticed at least that when I manually tweak the variables in the animation blueprint editor, the positions of the hands will change (finally!) after setting up the bones using the FABRIK nodes but I can't get that to happen in game using the hydra. I know this is somewhat confusing, but if you have any idea what I could be doing wrong, could you lend me a hand? I have the ability to post screenshots too but I didn't want to spam the forum with them just yet.
          Are you doing the calibrate step correctly? Stand in a t-pose (arms extended to the sides, shoulder height) and press the start button (or whatever button you used in your controller that the calibrate function is called from).
          Storyteller - An immersive VR audiobook player

          Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

          Comment


            Originally posted by getnamo View Post
            Second Step, inside the animation blueprint Event Graph, grab the values found in the pawn rotate them to align with the local axis (they use Y as forward for some reason).

            What's the definition for Convert Rotation? Thanks man, almost got this set up.
            Storyteller - An immersive VR audiobook player

            Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

            Comment


              Originally posted by n00854180t View Post
              What's the definition for Convert Rotation? Thanks man, almost got this set up.


              Also note that character owner has to be obtained and cast to your pawn type so that you can access the head/hand variables. I store this pointer as character owner so I can reference it later.
              Last edited by getnamo; 10-08-2014, 08:17 AM.
              Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

              Comment


                Originally posted by n00854180t View Post
                Are you doing the calibrate step correctly? Stand in a t-pose (arms extended to the sides, shoulder height) and press the start button (or whatever button you used in your controller that the calibrate function is called from).
                I have tried this. The result is still the same. I will double check my graphs again later today and see if maybe I can spot something with fresh eyes.

                EDIT: I think the errors I am getting have to do with the fact that I am trying to build on someone else's already established code in the case of the Rift Template. I've noticed that when I tried to switch the VR controller with the HydraPlayerController in the world settings, I began to encounter issues, such as errors coming up from all of the references to the VR controller in the character Blueprint. It may take me recreating all of those VR Controller variables in the Hydra Controller just to get those errors to subside. That is my theory anyway.
                Last edited by aialexander; 10-08-2014, 02:15 PM.

                Comment


                  Originally posted by aialexander View Post
                  I have tried this. The result is still the same. I will double check my graphs again later today and see if maybe I can spot something with fresh eyes.

                  EDIT: I think the errors I am getting have to do with the fact that I am trying to build on someone else's already established code in the case of the Rift Template. I've noticed that when I tried to switch the VR controller with the HydraPlayerController in the world settings, I began to encounter issues, such as errors coming up from all of the references to the VR controller in the character Blueprint. It may take me recreating all of those VR Controller variables in the Hydra Controller just to get those errors to subside. That is my theory anyway.
                  Did you just switch from one to the other?

                  You should probably Reparent one of them. There are multiple ways to set it up.


                  If you have a controller that already is a child of HydraPlayerController, you *could* make VRPlayerController a child of that.

                  Otherwise you could just make VRPlayerController a child of HydraPlayerController, then reimplement any of your Hydra related code. I suggest that way myself.

                  Here, here's some screenshots explaining.

                  Open VRPlayerController.

                  Click image for larger version

Name:	dvELjac.png
Views:	1
Size:	6.3 KB
ID:	1057626


                  Select File -> Reparent Blueprint.

                  Click image for larger version

Name:	VeVYai4.png
Views:	1
Size:	48.2 KB
ID:	1057627

                  Then select either HydraPlayerController (and subsequently implement the Hydra-related stuff into VRPlayerController, which will have access to the Hydra BP nodes), or select your already made PlayerController that implements the Hydra code.

                  Storyteller - An immersive VR audiobook player

                  Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

                  Comment


                    You can also use the HydraPluginActor, place it in the scene and get a pointer to the player controller in event begin play and use that reference inside the plugin actor. From there you forward your hydra input into variables in the player controller.

                    This was one of the reasons why I made a convenience actor, allowing you to extend functionality without changing any of the already implemented code. It is also how you can have control indirection, since the generic variables such as Left/Right Hand Position inside your player controller can be updated by different devices via plugins similar to the Hydra Plugin and the player controller doesn't care how the positions were obtained.
                    Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

                    Comment


                      Originally posted by n00854180t View Post
                      Did you just switch from one to the other?

                      You should probably Reparent one of them. There are multiple ways to set it up.


                      If you have a controller that already is a child of HydraPlayerController, you *could* make VRPlayerController a child of that.

                      Otherwise you could just make VRPlayerController a child of HydraPlayerController, then reimplement any of your Hydra related code. I suggest that way myself.

                      Here, here's some screenshots explaining.

                      Open VRPlayerController.

                      [ATTACH=CONFIG]13775[/ATTACH]



                      Select File -> Reparent Blueprint.

                      [ATTACH=CONFIG]13776[/ATTACH]

                      Then select either HydraPlayerController (and subsequently implement the Hydra-related stuff into VRPlayerController, which will have access to the Hydra BP nodes), or select your already made PlayerController that implements the Hydra code.

                      I gave this a shot by parenting the VR Controller to the HydraPlayerController and combining the code for the hydra that I had already programmed in another blueprint by recreating it in the VR Controller Blueprint and I managed to get the errors to stop. However, I still have not been able to get the character's arms to respond to the Hydra. I will keep trying and let you all know if I manage to come up with anything. If you happen to get that tutorial ready, I could really use it. Getnamo had a suggestion for me as well, so I'm going to look over his instructions again to see if I can understand it a little better.

                      Comment


                        I *almost* have the body setup working, but can't seem to get it so that if you lean back with the Rift, it will lean the torso correctly enough to prevent clipping through the head. I could just hide the head, but there would still be issues... I'd like to get it such that the head follows properly enough that you just don't see it (and it *almost* works well enough to do that now).

                        @Getnamo - How are you doing the Reach Rotation Yaw and Reach Alpha values? I assume you'd want to check if the previous IK didn't meet the target, but I don't see an obvious way of doing that from the IK nodes. Or are one or both just statically set?

                        Edit: I think I've improved the basic setup a little bit by using FABRIK instead of the Two-Bone IK for the "spine3" IK. I still have some problems right now, such as the HMD basically being rotated 90 degrees in terms of view compared to the player, but that should be fixable with the offset (which I haven't messed with). I have some other problems in my setup I need to fix.

                        Unfortunately due to how long this is taking to really get right, it's going to take me a bit longer to get the tutorial up.
                        Last edited by n00854180t; 10-11-2014, 08:03 PM.
                        Storyteller - An immersive VR audiobook player

                        Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

                        Comment


                          @n00854180t The reach is done the following way:



                          Again using a bunch of feel good variables, but it works

                          I *almost* have the body setup working, but can't seem to get it so that if you lean back with the Rift, it will lean the torso correctly enough to prevent clipping through the head. I could just hide the head, but there would still be issues... I'd like to get it such that the head follows properly enough that you just don't see it (and it *almost* works well enough to do that now).
                          You will always have to compromise on this a little if you use any form of animation. The third person character adds a head bob which 90% of the users don't want, but it makes for better external animation (I've tried attaching the camera to the head, its usable, but only for acclimated developers in vr, and generally prefer it without). For any high head bob motions where the user's real head doesn't bob you will need to have the head slightly desynced from the body, ideally we need a simple way to mask out the head. I'm thinking to maybe use a masked material with a variable that only triggers true for owner, either that or having split geometry (head/body separate, head ownersee = false). The advantage of the masked material is that this can be vertex painted (requiring no external geometry modification), whereas the separate geometry is probably better for performance. I haven't tested either so far.

                          Edit: I think I've improved the basic setup a little bit by using FABRIK instead of the Two-Bone IK for the "spine3" IK. I still have some problems right now, such as the HMD basically being rotated 90 degrees in terms of view compared to the player, but that should be fixable with the offset (which I haven't messed with). I have some other problems in my setup I need to fix.
                          Nice, curious to see this in action! One thing to remember for the HMD is that by default it is attached to your control rotation. Using oculus separate view code you regain the ability to rotate the hmd fully arbitrarily, which is what I use as a base. I believe the VR template uses a blueprint solution with counter rotations to the mesh, which I'm not a fan of. Unsure if this would impact your HMDOffset, but wanted to make sure you got your bases covered.


                          Also I've recently made an event-driven Leap Motion plugin which taught me some things as I experimented with interfaces and learned about C++ component registration. Will feed back some of that architecture back into the hydra plugin (convenience classes will remain unchanged functionally). This will mean being able to add a 'HydraComponent' to any blueprint and after adding an interface and setting self as delegate will allow you to receive hydra events to any blueprint. This would solve any re-parenting issues in the future and allow full flexibility of using input devices. This would for example, allow multiple input plugins that support this architecture, to be added to any blueprint and receive all of their events.
                          Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

                          Comment


                            Updated plugin to v0.7
                            -Updated to UE4.5
                            -New Component based system using interfaces; now able to extend any
                            blueprint class to have a Hydra component and receive the HydraInterface
                            events! See readme or unreal thread for details.
                            -All events now emit a UHydraSingleController, allowing for easy probe
                            of additional data at the event state.
                            -Some compatibility may be broken, but old style C++ extension is still
                            possible using the old delegate (still retained). New style C++ plugin
                            extension leads to less code, see convenience actor for details.

                            Component Based System
                            This method works by adding a Hydra Component and then subscribing to the events by adding a HydraInterface to your interfaces in your blueprint class. Both parts are required for you to receive hydra events.

                            Setup
                            Open the blueprint you wish to receive hydra events.

                            Add the Hydra Component to your blueprint through method 1 or 2.

                            (Option 1) Add component directly


                            (Option 2) Add component through event graph


                            To receive events we now have to add an interface. Click on Blueprint Props and under Details find 'Add' under interfaces. Add the HydraInterface.


                            How to Use

                            You can get any of the HydraInterface events by right clicking on the event graph and typing 'Hydra events'


                            An event will emit when the action occurs, giving all relevant data in context as pins in the node. You can filter the events by hand using Is x Hand function on the controller output.


                            The controller pointer is emitted for every relevant event, allowing you to conveniently access other variables you may be interested in that context. This removes the need to poll for most cases.


                            If you need to access past frames you can do so by querying your HydraComponent. The plugin stores the last 10 frames for use i.e. gesture recognition. The controllerId refers to each single controller, 0 or 1 is a valid input. You can also manually check if your hydra is available. Should the state change it will also emit an event since the plugin supports hot plugging.



                            Feedback is always welcome, seeing as this might break a few things, let me know if this setup makes things easier or harder for you

                            Edit:
                            Update to 0.7.1
                            -Synced folder layout with other plugins. To install you now need to copy both Binaries and Plugins for future plugin updates.
                            -Added support for 32bit shipping build, for either packaged or shipping build copy only the Binaries folder after packaging has been complete. See github instructions for details
                            Last edited by getnamo; 10-18-2014, 04:42 PM.
                            Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

                            Comment


                              Hey, Getnamo! I just wanted to let you know that I tried out your new system and went through this process for the new hydra plugin, but I seem to have broken something. Now every time I try to hit play in the editor, Unreal crashes. All I did was add the Hydra component to the default character blueprint. I will try the second option and see if I get the same result.

                              EDIT: Okay I've tried both methods and I've gotten crashes for both. Not sure what's causing it. For the time being I'll see what I can do with the older tutorials for this plugin and see how far I get.
                              Last edited by aialexander; 10-17-2014, 02:53 PM.

                              Comment


                                By the way, this may be slightly off topic, but would anyone be able to tell me what a poseable mesh is or how to go about creating one in Unreal? While reading a tutorial for the leap motion, I noticed this was one of the components, but I could not find any reference to what this would be in any of the Unreal documentation or forums.

                                Comment

                                Working...
                                X