Announcement

Collapse
No announcement yet.

"World to meters" parameter problem

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    [OCULUS] "World to meters" parameter problem

    Hello!

    I'm currently using the getnamo plugin for leap motion in unreal engine 4 in combination with occulus rift dk2 to make a game. The game will be played on a virtual table so the players should be able to reach everywhere on the map (it should feel like you are sitting around a real table). In order to achieve this I want to change the "world to meters" parameter in world settings under VR, but when i increase it to for example 200 (default is 100) the leap motion hands start behaving weird. A cross eye effect occurs and if I increase it even more the leap hands eventually disappears completely. One way to solve it is to keep the world to meters parameter as it is and just scale the game world down, but this resulted in issues with the navigation mesh as I had to scale everything down a lot.

    So my question is if this problem when increasing the "world to meters" parameter is at all connected to the leap motion plugin or is it some occulus/ue4 specific problem? Or am I missing something obvious here? Haven't found anyone with the same problem so im starting to suspect that it might be something fundamental im missing. It's my first time working with ue4, occulus and leap motion so everything is kind of new.

    #2
    So this issue has been partly solved. When changing the world to meters parameter you need to also change the IPD value by writing the console command "stereo e=<IPD value>". The default IPD is 0.064 and deafult world to meters is 100. So i tried changing the world to meters to 1000 and then IPD to 0.0064 which worked fine and the leap hands showed as normal. Then I tried world to meters = 10000 and IPD = 0.00064 and there is no cross eye effect on objects in the world when getting close so that issue is solved. However a new issue has arised: the leap hands starts freaking out, they wobble around like crazy. But at least they show up so it is a step in the right direction. If anyone has an idea as how to solve the "wobbling" hand issue i'd be glad to here!

    Comment


      #3
      Originally posted by Ders92 View Post
      So this issue has been partly solved. When changing the world to meters parameter you need to also change the IPD value by writing the console command "stereo e=<IPD value>".
      Thats interesting, I was alway's under the impression that world to metres scaled everything including the IPD.

      Comment


        #4
        Yes I thought so aswell, maybe thats something that has been fixed in 4.11? Im using 4.10 currently. Also I have looked through the source code for getnamos leap motion plugin and found the function that translates the hands (LeapHand.cpp). That function uses a function from Leapinterfaceutility.cpp: "FVector convertAndScaleLeapToUE(Leap::Vector leapVector)" and my thought is that the worldToMeters variable should be taken into account in that function.

        This is how the function looks like:

        FVector convertAndScaleLeapToUE(Leap::Vector leapVector)
        {
        //Scale from mm to cm (ue default)
        FVector vect = FVector(-leapVector.z * LEAP_TO_UE_SCALE,
        leapVector.x * LEAP_TO_UE_SCALE, leapVector.y * LEAP_TO_UE_SCALE);

        //Front facing leap adjustments
        if (LeapShouldAdjustForFacing)
        {
        vect = adjustForLeapFacing(vect);
        if (LeapShouldAdjustRotationForHMD)
        vect = adjustForHMD(vect);
        }
        return vect;
        }

        So im thinking something like this:
        FVector vect = FVector(-leapVector.z * LEAP_TO_UE_SCALE * (WorldToMeters/100.0),
        leapVector.x * LEAP_TO_UE_SCALE * (WorldToMeters/100.0), leapVector.y * LEAP_TO_UE_SCALE * (WorldToMeters/100.0));

        However im not sure if/how the c++ classes of getnamos plugin is connected to his blueprints because nothing happens when i make this change. Or maybe im thinking wrong here?

        Comment


          #5
          Originally posted by Ders92 View Post
          Yes I thought so aswell, maybe thats something that has been fixed in 4.11? Im using 4.10 currently. Also I have looked through the source code for getnamos leap motion plugin and found the function that translates the hands (LeapHand.cpp). That function uses a function from Leapinterfaceutility.cpp: "FVector convertAndScaleLeapToUE(Leap::Vector leapVector)" and my thought is that the worldToMeters variable should be taken into account in that function.

          This is how the function looks like:

          FVector convertAndScaleLeapToUE(Leap::Vector leapVector)
          {
          //Scale from mm to cm (ue default)
          FVector vect = FVector(-leapVector.z * LEAP_TO_UE_SCALE,
          leapVector.x * LEAP_TO_UE_SCALE, leapVector.y * LEAP_TO_UE_SCALE);

          //Front facing leap adjustments
          if (LeapShouldAdjustForFacing)
          {
          vect = adjustForLeapFacing(vect);
          if (LeapShouldAdjustRotationForHMD)
          vect = adjustForHMD(vect);
          }
          return vect;
          }

          So im thinking something like this:
          FVector vect = FVector(-leapVector.z * LEAP_TO_UE_SCALE * (WorldToMeters/100.0),
          leapVector.x * LEAP_TO_UE_SCALE * (WorldToMeters/100.0), leapVector.y * LEAP_TO_UE_SCALE * (WorldToMeters/100.0));

          However im not sure if/how the c++ classes of getnamos plugin is connected to his blueprints because nothing happens when i make this change. Or maybe im thinking wrong here?
          In theory that change would modify the position points of all the plugin position values since it is used everywhere in the plugin. That said I'm not sure what WorldToMeters actually does. If you can make an easily reproducible test project it might be easier to figure out what is going on under the hood.

          Try the change you suggested and compile the project with the plugin and it should use this new adjustment.
          Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

          Comment


            #6
            Originally posted by getnamo View Post
            In theory that change would modify the position points of all the plugin position values since it is used everywhere in the plugin. That said I'm not sure what WorldToMeters actually does. If you can make an easily reproducible test project it might be easier to figure out what is going on under the hood.

            Try the change you suggested and compile the project with the plugin and it should use this new adjustment.
            Thanks for your answer! I have tried the change I suggested (and recompiled everything in both visual studio and ue4 editor) but nothing happens. I tried with extreme values for worldToMeters (0.00001 and 1000000) and still nothing happens.

            If you want to see for yourself the issue you only have to create a new project and use the LeapEchoBasicHandRigged blue print (with gravityscale = 0 and max acceleration = 0 in charactermovement) as default pawn and then increase worldToMeters to 1000 and IPD to 0.0064. If you get the same issue I have then the hands will wobble/bounce weirdly and if you increase w2m to 10000 and ipd to 0.00064 then they start acting really strange and move with a delay and bounce around like crazy.
            Last edited by Ders92; 03-03-2016, 07:40 AM.

            Comment


              #7
              So I got stuff to happen now, had to rebuild the c++ project with the editor closed for the changes to take effect. It did not solve the problem to multiply with w2m as I suggested earlier, the hands just got offseted very far away from the camera and the wobbling issue still remained. I'm pushing my test project to github now but its taking forever, I can link to it when its done.

              Here is the github link to my test project. https://github.com/anduurs/LeapMotion_VR_Test.git

              I change the worldToMeters and IPD in my level blueprint
              Last edited by Ders92; 03-03-2016, 09:26 AM.

              Comment


                #8
                Originally posted by getnamo View Post
                In theory that change would modify the position points of all the plugin position values since it is used everywhere in the plugin. That said I'm not sure what WorldToMeters actually does. If you can make an easily reproducible test project it might be easier to figure out what is going on under the hood.

                Try the change you suggested and compile the project with the plugin and it should use this new adjustment.
                This is a video that illustrates the issue more clearly, with w2m = 10000 and ipd = 0.00064
                https://www.youtube.com/watch?v=V4st...ature=youtu.be

                Comment


                  #9
                  Originally posted by Ders92 View Post
                  This is a video that illustrates the issue more clearly, with w2m = 10000 and ipd = 0.00064
                  https://www.youtube.com/watch?v=V4st...ature=youtu.be
                  Interesting, the jitter I feel like is a precision error (floating point rounding error), I doubt ipd really supports values that low. I think someone from epic who has a better understanding of W2M does would be the right person to tell you how to handle what you're trying to do.
                  Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                  Comment


                    #10
                    Originally posted by getnamo View Post
                    Interesting, the jitter I feel like is a precision error (floating point rounding error), I doubt ipd really supports values that low. I think someone from epic who has a better understanding of W2M does would be the right person to tell you how to handle what you're trying to do.
                    Okey thanks for your answer! I forgot to add that in that video im using Orion, before things was even worse. But if ipd didnt support values this low then it feels like it shouldnt only be the leap hands behaving weird?

                    Do you have a suggestion on who I should contact? I have written to Chance Ivey who started the thread about the official leap motion plugin https://forums.unrealengine.com/show...-Motion-Plugin!

                    Comment


                      #11
                      Originally posted by getnamo View Post
                      Interesting, the jitter I feel like is a precision error (floating point rounding error), I doubt ipd really supports values that low. I think someone from epic who has a better understanding of W2M does would be the right person to tell you how to handle what you're trying to do.
                      I have been looking for where the updating of the hands is done and recently i have been looking in AnimBone.cpp but the translate method there only offsets the hands. And the LeapHand.cpp where i started looking only seems to scale the positions to ue4 coordinate space so my question is where is the code for the actual updating? It feels like it might be some kind of smoothness functionality within the updating code of the hands that might need to take account the increase in scale that w2m does. Also the ipd doesnt seem to have any effect on the jittering of the hands, it is only when increasing the w2m that the hands get unstable.

                      Also I wanna try to scale the size of the hands because they are pretty small in relation to the world, how do you do this? Doesnt seem to work to just scale them in the viewport
                      Last edited by Ders92; 03-11-2016, 09:39 AM.

                      Comment


                        #12
                        Originally posted by Ders92 View Post
                        I have been looking for where the updating of the hands is done and recently i have been looking in AnimBone.cpp but the translate method there only offsets the hands. And the LeapHand.cpp where i started looking only seems to scale the positions to ue4 coordinate space so my question is where is the code for the actual updating? It feels like it might be some kind of smoothness functionality within the updating code of the hands that might need to take account the increase in scale that w2m does. Also the ipd doesnt seem to have any effect on the jittering of the hands, it is only when increasing the w2m that the hands get unstable.

                        Also I wanna try to scale the size of the hands because they are pretty small in relation to the world, how do you do this? Doesnt seem to work to just scale them in the viewport
                        My recommendation for a start would be to use a custom setup. Get the LeapEventInterface hand moved events and scale them manually to w2m and output finger tips positions there first. If that works it may point at other issues.
                        Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                        Comment


                          #13
                          Originally posted by getnamo View Post
                          My recommendation for a start would be to use a custom setup. Get the LeapEventInterface hand moved events and scale them manually to w2m and output finger tips positions there first. If that works it may point at other issues.
                          Okey I will try it out! I also want to add that I tried out the built in plugin and it works way worse than yours in regards to orientation in hmd mode but it does not have the same issue with w2m, in other words no jittering or delayed movement when increasing w2m. So as I see it there are two possible solutions for this issue: either fix the weird orientation in the built in plugin or find a way to resolve the jittering and delayed movement issue in your plugin when increasing w2m. I looked through the source code for the built in plugin and they are indeed using w2m in the code that updates the bones which is why i have tried to do something similar in your plugin. However it is kind of difficult since the plugins differ very much in their approach to integrate Leap Motion in ue4. But i will try the custom setup as you suggested.

                          Comment


                            #14
                            So this issue has been solved now! The problem can be solved as follow:

                            Go into the cpp file in getnamos plugin called LeapInterfaceUtility.cpp and scroll down to the function called "convertAndScaleLeapToUE".
                            In this function you have to add the following:

                            float w2m = GWorld->GetWorldSettings()->AWorldSettings::WorldToMeters / 100.0f;
                            FVector vect = FVector(-leapVector.z * LEAP_TO_UE_SCALE * w2m, leapVector.x * LEAP_TO_UE_SCALE * w2m, leapVector.y * LEAP_TO_UE_SCALE * w2m);

                            This will make the leap hands take w2m into account and the jittering/delayed movement issue I experienced was removed. However the size of the hands does not get scaled which means you must do it manually.
                            To scale the hands manually I did the following:

                            1) export the "hands_lo" blueprint located in "LeapMotion Content/character/echo" as fbx file.
                            2) import the fbx file into blender and scale the skeleton with w2m (worldtometers/100).
                            3) then apply the scale and location of the skeleton and also apply for the mesh
                            4) select both of the skeleton and mesh and then export as .fbx , choose FBX 7.6 binary, check selected objects, check only armat, mesh and other (no idea if this stuff matters)
                            5) import the new fbx file to ue4 and make sure import as skeletal and import mesh is checked. Also make sure Use TOAs Ref Pose is checked (i got a strange orientation error when i didnt check it).
                            6) At last just delete the old hands_lo and its skeleton blueprint and force replace them with your new modified blueprints!

                            Comment


                              #15
                              Hello guys,

                              I'm working with the Leap Motion device and i got some scale issue that the method above don't fix.
                              I'm on Unreal Engine 4.17.0 compiled version from github using the LeapMotion Plugin mounter on an Oculus Rift CV1.
                              I would like to make my character as a geant with several "height" level (x3, x5, x10). For testing purpose, i'm working only with the x3 level => World to meters is set to 300.
                              I didn't get any cross eye effect or jittering of the hands, I guess this was fix in this version of Unreal.

                              I made the c++ fix in the 'convertAndScaleLeapToUE' function, and the movement are scaled, but not the hands.
                              I've tried to export 'hands_lo', scale on blender, and export the new asset (FBX binary 7.4 ). Removing 'hands_lo' and its skeleton forcing replace with my new assets, my hands are invisible, not displayed, in both normal W2M or W2M=300.0. Without any special output log.
                              Is there any other special settings for the export/import of the hands_lo.fbx ?

                              - Scaling the hands meshs, or the hands BP from viewport or in BP at runtime, the hands are well scaled but the movements are too large ( a 5cm translation of my real hand seems way bigger in the virtual world ~ 10/15cm ).
                              - If I remove the C++ w2m fix, and I just scale the hands, i got the good scale for hands but the movement are scaled down (as expected reading this thread).
                              => I don't really understand where / what is the coeff which multiply these movement scale from smaller to way bigger.

                              I tried to apply the same c++ w2m fix in the 'float ScaleLeapToUE(float LeapFloat)' but the issue is the same.


                              => Any ideas where this fbx import error come from ?
                              => Using the LeapRiggedEchoHands_Actor as base class, i never took a good look inside the plugin classes and content until now, and as I don't know much about skeletal mesh i'm not sure to understand why is there several assets for the hands mesh. Should i scale each echo_lo_r and echo_lo_l mesh instead of echo_lo . Why is there a "2 hands mesh" asset when both individual hand mesh are used in the BP. Is that only for animation ?

                              Thanks a lot for any help.

                              UPDATE:

                              I succeed to make it work with the Ders92's answer with few modifications :
                              1 ) I exported hands_lo_R and hands_lo_L as FBX
                              2 ) Scale in blender and export
                              3 ) Import in ue4
                              4) Make a copy of each assets of the hands : hands_lo_L, hands_lo_L_skeleton, hands_lo_R ...
                              5 ) Open the copied skeletal mesh asset for hands, change the path for the FBX source, then reimport using 'Reimport' button from the skeletal mesh window
                              6) Your new skeletal mesh can be set as skeletal mesh for your hands

                              => Modifying the Leap Plugin assets hands_lo "broke" the skeleton and hands doesn't appear. This method allow to have several scale of the hands without modiying the Leap plugin content.

                              This method works but requires some manual step, anyway this doesn't allow a progressive modification of the Worlds2Meters.
                              Doing the scale in runtime may allow to create proressive height modification, and more possibility for the height of the character.

                              Have you any idead if that possible to do that at runtime, without using a 3d software for scale the skeletal mesh ?





                              Last edited by Marcassin__; 08-30-2017, 05:12 AM.

                              Comment

                              Working...
                              X