Announcement

Collapse
No announcement yet.

Valve's SteamVR Input Plugin

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    [VIVE] Valve's SteamVR Input Plugin

    GitHub: https://github.com/ValveSoftware/steamvr_unreal_plugin
    Documentation: https://github.com/ValveSoftware/ste...al_plugin/wiki

    I'm wondering why there is no thread to discuss it.
    Marketplace: Vive Mocap Kit / Fingers Solver / VR IK Body Solver / Subtitles to LipSync / Dialogue System with Character Animation
    And random stuff at Youtube

    #2
    First of all, huge thanks to runeberg for his work.

    Here are my thoughts.
    (note: they're based on 0.5a version).

    1. I don't like actions binding policy. It actually started in the default in-engine SteamVR plugin. Generated manifest contains custom input actions created in the Project Settings.
    Example:

    Click image for larger version

Name:	svrp_001.png
Views:	1
Size:	36.0 KB
ID:	1614184

    Code:
       "actions": [
            {
                "name": "/actions/main/in/TeleportRight",
                "type": "boolean"
            },
            ...
        ]
    In fact, we don't need to update custom actions. It's enough to trigger internal action, in this case - "Motion Controller (R) Thumbstick". And UE4 is responsible for updating "TeleportRight".

    Code:
        "actions": [
            {
                "name": "/actions/main/in/MotionController_Right_Thumbstick",
                "type": "boolean"
            },
            ...
        ]
    That's all. It will work in all user-created actions binded to "Motion Controller (R) Thumbstick".

    2. We need input axes for fingers curl. No, not because I have such plugin on Marketplace Hand pose generated by SteamVR isn't suitable for all cases. I want (and I believe it's common situation) different poses for clenched fingers: for free hand (i. e. fist), for hand holding gun, for hand holding other tools. The easiest way to do that - interpolate fingers between open hand and manually selected grippnig pose. I need alpha value for interpolation, and finger curl works well for this purpose.

    3. Type of active motion controllers should be exposed to blueprints via functions library.

    Foe example, I use trackpad for locomotion with Vive Controllers, but I want to use joystick on Index Controllers, simply because they have trackpad with very limited X axis. So, I need to know type of controllers to make a decision.

    Code:
    FString USteamVRInputDeviceFunctionLibrary::GetActiveControllerType()
    {
        FString DeviceName = TEXT("undefined");
    
        // Get VRInput
        FSteamVRInputDevice* SteamVRInputDevice = GetSteamVRInputDevice();
        if (SteamVRInputDevice != nullptr && SteamVRInputDevice->SteamVRSystem != NULL)
        {
            // Iterate devices to find first active motion controller
            for (uint32 DeviceIndex = 0; DeviceIndex < vr::k_unMaxTrackedDeviceCount; ++DeviceIndex)
            {
                vr::ETrackedDeviceClass DeviceClass = SteamVRInputDevice->SteamVRSystem->GetTrackedDeviceClass(DeviceIndex);
                if (DeviceClass == vr::TrackedDeviceClass_Controller && SteamVRInputDevice->SteamVRSystem->IsTrackedDeviceConnected(DeviceIndex))
                {
                    char Output[vr::k_unMaxPropertyStringSize] = "";
    
                    // Get controller type
                    uint32 StringBytes = SteamVRInputDevice->SteamVRSystem->GetStringTrackedDeviceProperty(
                        (TrackedDeviceIndex_t)DeviceIndex,
                        ETrackedDeviceProperty::Prop_ModelNumber_String,
                        &Output[0],
                        k_unMaxPropertyStringSize);
    
                    DeviceName = *FString(ANSI_TO_TCHAR(Output));
    
                    break;
                }
            }
        }
        else
        {
            DeviceName = TEXT("vr_not_initialized");
        }
    
        return DeviceName;
    }
    Marketplace: Vive Mocap Kit / Fingers Solver / VR IK Body Solver / Subtitles to LipSync / Dialogue System with Character Animation
    And random stuff at Youtube

    Comment


      #3
      They assume that the default bindings remapping will be how you handle the difference of a Joystick / Trackpad, which is why they added duplicate hardcoded extra key inputs for every controller type supported by SteamVR. Those extra keys though are literally only used for creating the default mapping files and serve no other purpose, making it a bloated mess when trying to use their plugin alongside the oculus native setup or even general inputs. This was done to avoid the user having to map defaults in the SteamVR tool (which I had assumed was the entire point of it existing).

      I posted some suggestions that they scrap the extra duplicate keys for every controller, make a master list of key types (IE: SteamVR_Key_Trigger_Left) and have an array of default mapping structures that just lets you override keys that change between controllers to write out to the default mappings instead, an interface outside of the standard input config.

      Something like this but with SteamVR keys instead:



      As for the temporary keys, yeah they aren't needed assuming that people assign keys to actions, the only reason they exist AFAIK is if you have an action with no key assigned that you want to fire off in the steamvr plugin builds only, which could still be solved by assigning one of those SteamVR keys to it. There are notes in the OpenXR plugin that went out with 4.22 for Epic setting up the input system so that actions can be fired without a key assigned to them at all, which would straight up solve the issue cleanly when they get around to implementing it (openXR has the same action setup as the in engine steam vr beta input).


      For finger curls they exposed them to be BP accessible (and thus C++ too by calling the function library), though they are broken out into a struct with a separate float for each finger which is a bit annoying for general iteration, so you should already be good there.


      I totally agree on the controller type, their stance was that controller type shouldn't matter in the future since bindings and tracking quality can be used to infer differences, but there are specific situations where knowing the exact model is useful and it shouldn't be something we have to keep a separate library around for.

      *edit* they have a dev discord for it btw, you should have received an invite if you have dev hardware
      Last edited by mordentral; 05-01-2019, 12:46 PM.


      Consider supporting me on patreon

      My Open source tools and plugins
      Advanced Sessions Plugin
      VR Expansion Plugin

      Comment


        #4
        Situation with trackpad and joystic is an exception, and other exceptions are unlikely. There is not a lot of sence in building bulky architecture to deal with one exception. Moreover, it isn't possible to keep cross-controllers unification in a level of SteamVR actions/binding anyway (and let's not forget that, for the point of view of UE4, it also should be unified with another VR systems). Developers will need to deal with unification manually. One check more, one less - doesn't matter.

        Just another example: grabbing gun. Player with Vive controllers (old "wands") can't hold Grip button for a long time. I. e. natural setup is using one click (press-release) to take gun and then another click to throw. With Index, it makes sence to use grip press for grabbing and grip release for throwing. So we need one input event for Vive controllers and two separate events for Index.

        PS Didn't receive invite to discord with Knuckles btw. And a key for test content package either.
        Last edited by YuriNK; 05-01-2019, 02:39 PM.
        Marketplace: Vive Mocap Kit / Fingers Solver / VR IK Body Solver / Subtitles to LipSync / Dialogue System with Character Animation
        And random stuff at Youtube

        Comment


          #5
          That is more of a game logic change than an input change though.

          Both IndexGrip and std controller grip events can throw Pressed/Released, and different objects in game will work better with different methods of grip/release. You can also make entirely separate actions (and action sets) if you choose, I don't see how assigning specific keys is any better with that example.

          Regardless, with the OpenXR plugin taking the same path for input (steamVRs input is modeled around OpenXR's)....this is likely how the unification of input will end up anyway across headsets.


          Consider supporting me on patreon

          My Open source tools and plugins
          Advanced Sessions Plugin
          VR Expansion Plugin

          Comment


            #6
            Hi YuriNK !

            Thanks for the feedback.


            Re: Action Bindings

            One of the benefits of the Action Bindings is actually to support the scenario you cited in Item #3 (Trigger an action via a different input path depending on controller type).

            In the samples that came with the plugin, you can see us demo this in most, if not all of the sample inputs. For instance, in the sample maps, pressing the Trackpad in the Index Controller will trigger the Teleport Action, but if you have an Oculus Touch Controller, you will trigger the Teleport by pressing the Joystick - all achievable without additional developer logic in their game/experience.

            As Mordentral mentioned, the Action Bindings approach also conforms with upcoming OpenXR standards, and my understanding is that Epic is internally doing a refactor to support this standard. Our implementation is to help bridge this gap in the meantime as well as make sure that developers using older versions of the Engine still can get these benefits, as we understand upgrading Engine versions can at times be quite a big ask for some studios and/or experiences/games.


            Re: Finger Curls

            We do have support for Finger Curls and Splays in the plugin via the node "Get Finger Curls and Splays". We have demoed how you can use this in the sample maps - if you do a "peace" sign, the JoeJeff character will jump. This is done via a rudimentary gesture recognition using finger curls.



            Re: Trigger an action via a different input path

            See my response in Re: Action Bindings.

            In addition, the Action Bindings paradigm is also meant to allow users of your game or experience to remap input as they see fit without additional logic from the developer. For instance, if I prefer using my Left Hand to do "Use" command (Trigger) in Trevor Saves the Universe, I can easily remap it in the SteamVR Dashboard without the developer having to expose UI and implement logic for it. With SteamVR, I can then go further, and share this input scheme to other left-handed folks who plays the game and might be interested.


            Re: Abstracted Motion Controller keys (e.g. MotionController)

            We have left support for this. If there's no custom "SteamVR Input {Manufacturer} {Controller} {Key}" defined in the mappings, and the Engine's MotionController abstraction is mapped instead, then the plugin will generate the controller bindings for ALL supported controller types.

            For backwards compatibility, the MotionController mapping is ignored for the auto-generation of the bindings by the plugin as long as there is at least one SteamVR Input key mapped.


            Re: Controller Types

            Ideally this shouldn't be needed and only for cases related to specific inputs. In the plugin, one of our experimental nodes in the latest release is the node "Get Origin Localized Name", using this, you can query, input-per-input what controller type triggered the action even down to which specific key was used by your player.

            Outside this level of granularity, outside OpenXR, the only (I believe) current need for checking Controller Types at the moment, is when trying to determine position & rotation offsets for various controllers that's unfortunately not (yet) supported by our pose actions.


            Re: Knuckles Discord Server

            Yes, we maintain a Discord server. Please send an email to your Valve contact for the Knuckles requesting access. Failing that, send me a DM of your details (email) you used to request the Knuckles and I can forward the request on internally to the server admin. No guarantees though, as they are understandably quite busy at the moment.

            I hope the above information helps.

            Cheers,
            Rune.
            Last edited by RuneBerg; 05-01-2019, 05:16 PM.
            ---
            runeberg
            VR Developer (C++ / BP), Room-scale specialist


            Web: http://runeberg.io
            GitHub: https://github.com/1runeberg/RunebergVRPlugin
            Free VR Plugin: https://www.unrealengine.com/marketp...ponents-plugin

            Udacitty Unreal Nanodegree Mentor & Project Reviewer: https://www.udacity.com/course/learn...dations--nd117


            Comment


              #7
              Originally posted by mordentral View Post
              That is more of a game logic change than an input change though.
              So why a choise between input buttons on different controllers isn't game logic? We actually can use long holding of grip button with Vive controller. The same way, we can bind locomition to trackpad on Knuckles. Both decisions are bad for gameplay, but possible. There is no need to push one of this problems away from game logic to input logic.
              Last edited by YuriNK; 05-02-2019, 05:01 AM.
              Marketplace: Vive Mocap Kit / Fingers Solver / VR IK Body Solver / Subtitles to LipSync / Dialogue System with Character Animation
              And random stuff at Youtube

              Comment


                #8
                runeberg

                I see your point.

                Default SteamVR plugin is bad. It should be replaced. But if you want to do that, transition to your plugin should be very smooth for developers (especially inexperienced, and they're majority). I have a plugin using Vive Trackers on Marketplace. In 4.22, customers experience problems with motion controllers components in non-VR mode (thougth, I didn't test it yet 4.22.1). And I can't recommend them your plugin, which solves this problem, yes, but also could break their input.

                I would suggest to refactor the input system like Mordentral said, but a bit differently: use general "Motion Controller (R/L)" actions when they fit. Add advanced, but also generalized actions (not tied to controller type) for lacking inputs of Knuckles. And create custom UDataAsset (can be applied in the plugin's settings) to allow developer to override binding for different controller types. Like this:
                • Motion Controller (R) Thumbstick: [Vive Controller -> Trackpad] [Valve Index -> Joystick].
                • ...
                Then, use this information to create different binding files. I thnik it's easier then modifying Project Settings window or creating new editor for bindings. Plus, it would allow developers to switch between different bindings. And "Motion Controller (R/L)" actions would always work.

                As for the discord, unfortunately I don't have enough free time and so not very interested in joining discussion. I modified the plugin for my purposes and decided to share my thougth. I hope, this feedback will help you in development. Good luck!
                Last edited by YuriNK; 05-02-2019, 02:48 AM.
                Marketplace: Vive Mocap Kit / Fingers Solver / VR IK Body Solver / Subtitles to LipSync / Dialogue System with Character Animation
                And random stuff at Youtube

                Comment


                  #9
                  Originally posted by YuriNK View Post
                  runeberg

                  I see your point.

                  Default SteamVR plugin is bad. It should be replaced. But if you want to do that, transition to your plugin should be very smooth for developers (especially inexperienced, and they're majority). I have a plugin using Vive Trackers on Marketplace. In 4.22, customers experience problems with motion controllers components in non-VR mode (thougth, I didn't test it yet 4.22.1). And I can't recommend them your plugin, which solves this problem, yes, but also could break their input.

                  I would suggest to refactor the input system like Mordentral said, but a bit differently: use general "Motion Controller (R/L)" actions when they fit. Add advanced, but also generalized actions (not tied to controller type) for lacking inputs of Knuckles. And create custom UDataAsset (can be applied in the plugin's settings) to allow developer to override binding for different controller types. Like this:
                  • Motion Controller (R) Thumbstick: [Vive Controller -> Trackpad] [Valve Index -> Joystick].
                  • ...
                  Then, use this information to create different binding files. I thnik it's easier then modifying Project Settings window or creating new editor for bindings. Plus, it would allow developers to switch between different bindings. And "Motion Controller (R/L)" actions would always work.

                  As for the discord, unfortunately I don't have enough free time and so not very interested in joining discussion. I modified the plugin for my purposes and decided to share my thougth. I hope, this feedback will help you in development. Good luck!
                  Appreciate your feedback, I'll bring yours as well as other developers input so far to our internal discussions.

                  Thanks YuriNK!
                  Last edited by RuneBerg; 05-02-2019, 11:35 PM.
                  ---
                  runeberg
                  VR Developer (C++ / BP), Room-scale specialist


                  Web: http://runeberg.io
                  GitHub: https://github.com/1runeberg/RunebergVRPlugin
                  Free VR Plugin: https://www.unrealengine.com/marketp...ponents-plugin

                  Udacitty Unreal Nanodegree Mentor & Project Reviewer: https://www.udacity.com/course/learn...dations--nd117


                  Comment


                    #10
                    We just released v0.7a of the SteamVR UE4 Plugin: Debugging enhancements & various fixes.
                    Version/s Affected Change/Fix description
                    All Upgrade OpenVR to 1.3.22
                    All Expose GetOriginTrackedDeviceInfo in BP
                    All Temporary keys are stored in the DefaultInput.ini when remappings are done via the UI. This is normally fin, but in cases of major refactors in the mapping scheme, it can lead to input breakage.

                    Thanks @SquanchGames (Alex/SparkyMcSparks) for reporting.
                    All Node: Experimental node "Get Get SteamVR Action Array" is broken in 0.65a
                    All Fix error in sample content when using the Bow and Arrow
                    All Add back error log when a skeletal handle is not found or can’t be found in the running project.
                    4.16+ Fix sample content warnings in higher versions of the engine (1M Cube, ProjectPointToNavigation Deprecation warning)
                    All New nodes useful for testing/debugging input (documentation in SteamVRDeviceFunctionLibrary.h):
                    • GetSteamVR_ActionArray
                    • FindSteamVR_Action
                    • GetSteamVR_ActionSetArray
                    • GetSteamVR_OriginLocalizedName
                    • ShowSteamVR_ActionOrigin
                    • FindSteamVR_ActionOrigin
                    All X,Y Buttons for Oculus Touch aren’t being generated properly (left/right hand)
                    All Buttons that are mapped to an axis action are set to "force" instead of "scalar_constant"

                    Thanks Smapty (Karl Johnson) for reporting
                    All Vive Trackpad Touch Bindings (UE4 Side) become Press/Click bindings (SteamVR Input Side)

                    Thanks Smapty (Karl Johnson) for your contribution to the plugin
                    All Fix for Issue #11: Ensure capsense is recognized properly Throw an error in the log when we don’t see pose/skeleton actions for the controllers.
                    Special thanks to Squanch Games & Cloud Gate Studios for helping us test various fixes that helped in this release.

                    If you need help integrating in your own title or test future versions, let me know!

                    Cheers,
                    Rune.
                    ---
                    runeberg
                    VR Developer (C++ / BP), Room-scale specialist


                    Web: http://runeberg.io
                    GitHub: https://github.com/1runeberg/RunebergVRPlugin
                    Free VR Plugin: https://www.unrealengine.com/marketp...ponents-plugin

                    Udacitty Unreal Nanodegree Mentor & Project Reviewer: https://www.udacity.com/course/learn...dations--nd117


                    Comment


                      #11
                      We've hit 1.0 and just released the plugin in the Marketplace!

                      https://steamcommunity.com/app/25082...1595050750522/

                      Comment


                        #12
                        Thanks - this is awesome work as it really helps support development on a wide array of hardware configurations!
                        David Gillespie
                        Associate
                        Foster + Partners

                        Comment


                          #13
                          I said that and then realised that it adds a whole lot more options to the motion controllers motion source, apparently removing "Left" and "Right" from the options. Has this been changed to something else as I've been struggling to get this to work with a Mixed Reality controller?

                          Click image for larger version  Name:	MotionControllerSettings.png Views:	2 Size:	53.8 KB ID:	1630377
                          David Gillespie
                          Associate
                          Foster + Partners

                          Comment


                            #14
                            Originally posted by david.gillespie View Post
                            I said that and then realised that it adds a whole lot more options to the motion controllers motion source, apparently removing "Left" and "Right" from the options. Has this been changed to something else as I've been struggling to get this to work with a Mixed Reality controller?

                            Click image for larger version Name:	MotionControllerSettings.png Views:	2 Size:	53.8 KB ID:	1630377
                            Hi David - Thanks for the feedback.

                            Those extra MotionSources you are seeing is not coming from our plugin. If I'm not mistaken, that's coming from the MagicLeap plugin that's enabled by default in 4.22 at least.

                            Cheers.

                            Comment


                              #15
                              Thanks for letting me know. Have managed to get it working following your wiki in more detail. Seems to have been a bug on my part with the motion controller setup so apologies there.

                              https://github.com/ValveSoftware/ste...iki/Quickstart

                              I assume now with the marketplace install there is no need to convert a BP project to C++?

                              It would be great to understand what the benefits are of jumping through the hoops required for SteamVR input as I'm having to create input actions in UE4 then generate all the control config data through 3 buttons in the plugin and then map them again in the Steam config. Willing to do this to allow for expanded controller support but it would be great to be able to streamline this.

                              Also support for combinations of button presses would be great too. It would be a bit easier (and I guess better for performance) if it could be handled lower level.
                              David Gillespie
                              Associate
                              Foster + Partners

                              Comment

                              Working...
                              X