Announcement

Collapse
No announcement yet.

Razer Hydra Plugin

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    #91
    @Getnamo, for the case where you want to use the Hydras to control the hands, how do you set that up to get from base-station-relative to something that's usable as an IK target?

    My current idea is to do something like this:

    C = inverse(A) * B

    Where A is the bone transform for the hand, B is the transform for the Hydra, and C is the bone-transform relative Hydra transform. Does that make sense at all? I'm working in Blueprint btw.
    Storyteller - An immersive VR audiobook player

    Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

    Comment


      #92
      @Getnamo

      Hello again!

      While I'm not sure what I did right this time, I took your advice and created a new project and re-copied the "Plugins" folder to my project folder and it worked. All of the events were there inside the HydraPluginActor Blueprint. I'm not sure what exactly caused that problem to begin with. That I know of, I did not do anything differently. I know for certain that the plugin was enabled when I first tried it out because I found the HydraPluginActor immediately when I searched in the Class Viewer. In any case, thank you for your response!

      Originally posted by n00854180t View Post
      @Getnamo, for the case where you want to use the Hydras to control the hands, how do you set that up to get from base-station-relative to something that's usable as an IK target?

      My current idea is to do something like this:

      C = inverse(A) * B

      Where A is the bone transform for the hand, B is the transform for the Hydra, and C is the bone-transform relative Hydra transform. Does that make sense at all? I'm working in Blueprint btw.
      I would also be interested in learning this. Ideally, I would like to set up a pair of hands in Unreal that will respond the motion of the Hydra and will have the ability to interact with objects in the scene. However, because I am not a programmer, I am uneducated in this process. So far, this forum has been my only resource to find these answers. I have seen plenty of examples of this working in Unity, but not as many in Unreal, and those that have succeeded in implementing this functionality that I have reached out to have not yet been able respond to my questions. If anyone has knowledge of this process that would be willing to share it with me I would greatly appreciate a push in the right direction.

      Comment


        #93
        Originally posted by aialexander View Post
        Hello again!

        While I'm not sure what I did right this time, I took your advice and created a new project and re-copied the "Plugins" folder to my project folder and it worked. All of the events were there inside the HydraPluginActor Blueprint. I'm not sure what exactly caused that problem to begin with. That I know of, I did not do anything differently. I know for certain that the plugin was enabled when I first tried it out because I found the HydraPluginActor immediately when I searched in the Class Viewer. In any case, thank you for your response!

        ...

        I would also be interested in learning this. Ideally, I would like to set up a pair of hands in Unreal that will respond the motion of the Hydra and will have the ability to interact with objects in the scene. However, because I am not a programmer, I am uneducated in this process. So far, this forum has been my only resource to find these answers. I have seen plenty of examples of this working in Unity, but not as many in Unreal, and those that have succeeded in implementing this functionality that I have reached out to have not yet been able respond to my questions. If anyone has knowledge of this process that would be willing to share it with me I would greatly appreciate a push in the right direction.
        My guess is you missed placing the actor in the scene, but I'm glad its working for you now . Regarding the second part, see my answer below or if you're patient VR Motion plugin will eventually be released which will have this setup for the hydra as a default down the line.

        Originally posted by n00854180t View Post
        @Getnamo, for the case where you want to use the Hydras to control the hands, how do you set that up to get from base-station-relative to something that's usable as an IK target?

        My current idea is to do something like this:

        C = inverse(A) * B

        Where A is the bone transform for the hand, B is the transform for the Hydra, and C is the bone-transform relative Hydra transform. Does that make sense at all? I'm working in Blueprint btw.
        In the actor where you wish to use IK, find where the root component origin is. Translate Hydra Base to Origin and all your position data will be now in component space (actor local space). You can then translate this easily to the bone space, or you can use this directly by selecting the bone position relative to component space.

        To translate the base to origin I did the following: 1) require a T-pose calibration, center point of both hands at this pose is the shoulder mid point. 2) Add shoulder to origin offset; using a hardcoded Z(height) offset is OK for most use cases, will need another data point to provide full flexibility, but to my knowledge torso lengths do not vary by a massive amount and can easily be corrected by the user by calibrating with arms a little higher or lower. In my case the origin was near the waist so it was something like 40cm down from shoulder midpoint. Essentially you are getting a vector pointing from base -> player's actor origin. All hydra positions with this correction will now report the distance from that point.

        While I am planning to have this included automatically as a calibration class in VR Motion plugin, my current work load is unfortunately delaying the plugin, so I hope this will help you get your hands working in the meantime!
        Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

        Comment


          #94
          Originally posted by getnamo View Post
          My guess is you missed placing the actor in the scene, but I'm glad its working for you now . Regarding the second part, see my answer below or if you're patient VR Motion plugin will eventually be released which will have this setup for the hydra as a default down the line.



          In the actor where you wish to use IK, find where the root component origin is. Translate Hydra Base to Origin and all your position data will be now in component space (actor local space). You can then translate this easily to the bone space, or you can use this directly by selecting the bone position relative to component space.

          To translate the base to origin I did the following: 1) require a T-pose calibration, center point of both hands at this pose is the shoulder mid point. 2) Add shoulder to origin offset; using a hardcoded Z(height) offset is OK for most use cases, will need another data point to provide full flexibility, but to my knowledge torso lengths do not vary by a massive amount and can easily be corrected by the user by calibrating with arms a little higher or lower. In my case the origin was near the waist so it was something like 40cm down from shoulder midpoint. Essentially you are getting a vector pointing from base -> player's actor origin. All hydra positions with this correction will now report the distance from that point.

          While I am planning to have this included automatically as a calibration class in VR Motion plugin, my current work load is unfortunately delaying the plugin, so I hope this will help you get your hands working in the meantime!
          Ohhhhh okay, that explains your code snippet from before. I didn't realize that's how you were doing it. For whatever reason at the time I just figured it'd be better to do it more correctly math-wise and try and get the hydra position into bone space.

          Anyway, I'll give that a try, thanks

          Edit: So I suppose after calculating the offset, I just want to subtract it from any reported hydra positions?
          Last edited by n00854180t; 10-02-2014, 04:12 AM.
          Storyteller - An immersive VR audiobook player

          Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

          Comment


            #95
            Originally posted by n00854180t View Post
            Ohhhhh okay, that explains your code snippet from before. I didn't realize that's how you were doing it. For whatever reason at the time I just figured it'd be better to do it more correctly math-wise and try and get the hydra position into bone space.

            Anyway, I'll give that a try, thanks

            Edit: So I suppose after calculating the offset, I just want to subtract it from any reported hydra positions?
            The problem with the math approach is that you do not have a reference to where the player is sitting/standing in reference to the base. You cannot escape needing a calibration unless you have a controller as a body-fixed point (which you will be able to do with a 3+ point STEM system). If you're using the hydras as hands, they will both be movable points in relation to the body so you need to get a vector to the players origin to adjust for base position variance (real world table heights and offsets).

            The other thing to consider is that different players have different arm lengths and while you would think a direct 1:1 mapping would be ideal, the avatar people embody may be larger/smaller than they are in real life and you will need to scale hydra movement to be in sync with the avatar. A T-pose handles both of these things which is why I consider it the ideal calibration for Hydras.

            In terms of the offset, it depends on how you are getting your vectors (adding or subtracting), but it should be RawHydraPosition + BaseToShoulderVector + ShoulderToOriginOffset = HydraInOrigin. If you are using world positions remember that all of these are relative and you will need to subtract out the actor world position from the instances you use world position.

            Just wanted to add that one of the best ways to debug vectors is to draw them (Draw Debug Arrow). One of the very cool things about rift development is that you can see these vectors in true 3d, so you can visualize the whole setup to ensure every vector is doing the correct thing.
            Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

            Comment


              #96
              Originally posted by getnamo View Post
              The problem with the math approach is that you do not have a reference to where the player is sitting/standing in reference to the base. You cannot escape needing a calibration unless you have a controller as a body-fixed point (which you will be able to do with a 3+ point STEM system). If you're using the hydras as hands, they will both be movable points in relation to the body so you need to get a vector to the players origin to adjust for base position variance (real world table heights and offsets).

              The other thing to consider is that different players have different arm lengths and while you would think a direct 1:1 mapping would be ideal, the avatar people embody may be larger/smaller than they are in real life and you will need to scale hydra movement to be in sync with the avatar. A T-pose handles both of these things which is why I consider it the ideal calibration for Hydras.

              In terms of the offset, it depends on how you are getting your vectors (adding or subtracting), but it should be RawHydraPosition + BaseToShoulderVector + ShoulderToOriginOffset = HydraInOrigin. If you are using world positions remember that all of these are relative and you will need to subtract out the actor world position from the instances you use world position.

              Just wanted to add that one of the best ways to debug vectors is to draw them (Draw Debug Arrow). One of the very cool things about rift development is that you can see these vectors in true 3d, so you can visualize the whole setup to ensure every vector is doing the correct thing.
              Sorry to keep bugging you about this, but I just wanna clear up some stuff in your equation there.

              BaseToShoulderVector = Vector from the base station to the shoulder mid point?

              ShoulderToOriginOffset = Shoulder mid point to character origin?
              Storyteller - An immersive VR audiobook player

              Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

              Comment


                #97
                Originally posted by getnamo View Post

                In the actor where you wish to use IK, find where the root component origin is. Translate Hydra Base to Origin and all your position data will be now in component space (actor local space). You can then translate this easily to the bone space, or you can use this directly by selecting the bone position relative to component space.

                To translate the base to origin I did the following: 1) require a T-pose calibration, center point of both hands at this pose is the shoulder mid point. 2) Add shoulder to origin offset; using a hardcoded Z(height) offset is OK for most use cases, will need another data point to provide full flexibility, but to my knowledge torso lengths do not vary by a massive amount and can easily be corrected by the user by calibrating with arms a little higher or lower. In my case the origin was near the waist so it was something like 40cm down from shoulder midpoint. Essentially you are getting a vector pointing from base -> player's actor origin. All hydra positions with this correction will now report the distance from that point.
                I hope you will forgive me, but I don't understand what this means. Is this done with the character Blueprint, or is this done using C++? Am I correct to assume that the shoulder midpoint of the chosen actor will be the Hydra Base or does the base refer to something else? Is this done by dropping the Hydra Plugin actor into the Character Blueprint? Is the Shoulder to Origin offset added using a Blendspace? Is there a kind soul out there that would be willing to give me a breakdown of this? I am unsure of where in the editor to look first.
                Last edited by aialexander; 10-02-2014, 03:27 PM.

                Comment


                  #98
                  Originally posted by n00854180t View Post
                  Sorry to keep bugging you about this, but I just wanna clear up some stuff in your equation there.

                  BaseToShoulderVector = Vector from the base station to the shoulder mid point?

                  ShoulderToOriginOffset = Shoulder mid point to character origin?
                  Whoops! Made direction mistakes in the definitions I gave you, it should be

                  RawHydraPosition - BaseToShoulderVector - ShoulderToOriginOffset = HydraInOrigin

                  see this masterpiece :



                  This can of course be rewritten as:
                  RawHydraPosition + ShoulderToBaseVector + OriginToShoulder = HydraInOrigin

                  NB: General way to visualize adding vectors



                  if you subtract them simply reverse the direction of the B vector.
                  Attached Files
                  Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

                  Comment


                    #99
                    Originally posted by aialexander View Post
                    I hope you will forgive me, but I don't understand what this means. Is this done with the character Blueprint, or is this done using C++? Am I correct to assume that the shoulder midpoint of the chosen actor will be the Hydra Base or does the base refer to something else? Is this done by dropping the Hydra Plugin actor into the Character Blueprint? Is the Shoulder to Origin offset added using a Blendspace? Is there a kind soul out there that would be willing to give me a breakdown of this? I am unsure of where in the editor to look first.
                    I plan to make an example of how to set this up once I get it set up myself, if you're willing to wait a couple of days.

                    Originally posted by getnamo View Post
                    Whoops! Made direction mistakes in the definitions I gave you, it should be

                    RawHydraPosition - BaseToShoulderVector - ShoulderToOriginOffset = HydraInOrigin

                    see this masterpiece :



                    This can of course be rewritten as:
                    RawHydraPosition + ShoulderToBaseVector + OriginToShoulder = HydraInOrigin

                    NB: General way to visualize adding vectors



                    if you subtract them simply reverse the direction of the B vector.


                    Perfect, thanks dude, this is really helpful. I think I should be able to get it set up in the next day or two using the diagram and other info here.
                    Storyteller - An immersive VR audiobook player

                    Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

                    Comment


                      Originally posted by aialexander View Post
                      I hope you will forgive me, but I don't understand what this means. Is this done with the character Blueprint, or is this done using C++? Am I correct to assume that the shoulder midpoint of the chosen actor will be the Hydra Base or does the base refer to something else? Is this done by dropping the Hydra Plugin actor into the Character Blueprint? Is the Shoulder to Origin offset added using a Blendspace? Is there a kind soul out there that would be willing to give me a breakdown of this? I am unsure of where in the editor to look first.
                      This is purely referring to the vector math required to translate the raw hydra positions (what you get from Hydra Moved event/etc) to actor space.

                      A T-pose means you ask the user to spread out their arms forming a T (can be standing or sitting). You will capture the position that the hydras report at this point, for example by asking them to push a button when they hold this pose, then save the Left and Right hydra positions that you get when they do this.

                      If you then do (Left Hydra Raw + Right Hydra Raw)/2, you get the shoulder midpoint position of your user in hydra space. You can now use this calibration point to translate all your future input into actor space.

                      This is achieved by
                      RawHydraPosition - BaseToShoulderVector - ShoulderToOriginOffset = HydraInOrigin

                      All of this is simply vector addition and division which you can implement in your blueprint.

                      You can then use the HydraInOrigin position vector to drive your IK setup.

                      Originally posted by n00854180t
                      I plan to make an example of how to set this up once I get it set up myself, if you're willing to wait a couple of days.
                      Very cool, looking forward to seeing your implementation!
                      Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

                      Comment


                        Originally posted by n00854180t View Post
                        I plan to make an example of how to set this up once I get it set up myself, if you're willing to wait a couple of days.
                        Thank you! That would be very much appreciated! And thank you also, Getnamo, for your quick and detailed response. I think I have a basic understanding of how this would be done. Now I would just need to see it implemented, as n00854180t seems to be currently working on. I appreciate your patience with me as I am still new to Blueprints and coding in Unreal as well as using the Hydra. You have been a great help so far!

                        Comment


                          @Getnamo, no rush on these questions, but I am curious.

                          I'm using the -1 * (RawInputleft + Right)/2 method, and subtracting the offsets.

                          Right now, I have to swap X and Y, and also negate the raw Y value. I tried debug rendering but didn't manage to get it working before I got the actual hand motion *basically* working.

                          I'm also having to add some to the Z position after doing the offset subtractions, by about 100 units.

                          Anyway, thanks a lot for your help, I'm going to keep banging on this and tweaking it up this weekend and will release some video and a sample soon.
                          Storyteller - An immersive VR audiobook player

                          Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

                          Comment


                            Fixed the last of my issues - the offset upwards I was having to add was my fault, I had a number backwards somewhere. It seems to work quite well, though I'll need to get the Oculus Config Utility height value to better match the character to the player. The character is shorter than me right now I think, so my hand ends up about 1ft below where my RL hand is.

                            Anyway, many thanks Getnamo for your help, I'm going to hook up the rest of the stuff and get an example ported over into the VR Game Template soon.
                            Storyteller - An immersive VR audiobook player

                            Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

                            Comment


                              Can't wait to see it!

                              Comment


                                Originally posted by aialexander View Post
                                Can't wait to see it!
                                I'm gonna dump some screenshots of the blueprints on ya, they are very simple so it should be quite easy to replicate. I can't go into super detail on each right at the moment but feel free to ask questions and I'll get to them at some point. I still plan on porting into the VR Game Template soon.

                                Edits incoming.

                                Copy the Hydra Plugin into your project dir and enable it in the Plugins menus, restart.

                                Make a new BP derived from HydraPlayerController. I called mine DHydraPlayerController, D being for Dungeon (Survival).

                                Go to World Settings and set the game mode override settings to use your HydraPlayerController BP you just made.

                                Click image for larger version

Name:	Ca3Vq5N.png
Views:	1
Size:	14.2 KB
ID:	1057436

                                This is in the Hydra player controller. Notice the variables there and the function Calibrate Hydra (I use start, easy to find on the controller). The base offset one is just a static 40 in Z, it doesn't change. I could use a float but I plan on doing a more thorough calibration step in the future to get more exact mappings, so I'm leaving it as a vector for now. The midpoint we calculate.

                                Click image for larger version

Name:	pLsQj7a.png
Views:	1
Size:	112.5 KB
ID:	1057437

                                Next comes the Calibrate function, also in the controller:

                                Click image for larger version

Name:	LrEhhOD.png
Views:	2
Size:	100.9 KB
ID:	1057438


                                Next we're going into the character's animation blueprint

                                Click image for larger version

Name:	nNDvRJP.png
Views:	1
Size:	16.1 KB
ID:	1057439

                                First, add these variables

                                Click image for larger version

Name:	ukCNya0.png
Views:	1
Size:	3.5 KB
ID:	1057440

                                Then create this (looks messy but is simple). Pull off the character node to get the controller.

                                Click image for larger version

Name:	NOETq7k.png
Views:	1
Size:	312.6 KB
ID:	1057441

                                The last step is to set up something like this in the AnimGraph. Ignore the wiring that goes off screen, it's just doing a static rotation to fix the default hands for my particular character.

                                Click image for larger version

Name:	X5rBQ7o.png
Views:	1
Size:	245.3 KB
ID:	1057442

                                Didn't notice those nodes were selected in that screenshot - there's no significance to it.


                                I'm using component space for the IK target positions.
                                Last edited by n00854180t; 10-04-2014, 09:38 PM.
                                Storyteller - An immersive VR audiobook player

                                Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

                                Comment

                                Working...
                                X